[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:55:48 dist-test-slave-21b9 krb5kdc[30680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:55:48 dist-test-slave-21b9 krb5kdc[30680](info): set up 2 sockets
Aug 09 04:55:48 dist-test-slave-21b9 krb5kdc[30680](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 04:55:49.120304 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:55:49 dist-test-slave-21b9 krb5kdc[30680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:55:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:55:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:55:49.308992 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:43611
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:43611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:55:50.054214 30707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:55:50.054859 30707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:55:50.055156 30707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:55:50.066363 30707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:55:50.066641 30707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:55:50.066840 30707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:55:50.067009 30707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:55:50.067144 30707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:55:50.083424 30707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:43611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:43611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:55:50.086005 30707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:55:50.088231 30707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:55:50.114012 30720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:50.113893 30718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:50.116034 30726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:55:50.117550 30707 server_base.cc:1034] running on GCE node
I20260809 04:55:50.118983 30707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:55:50.121819 30707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:55:50.123163 30707 hybrid_clock.cc:648] HybridClock initialized: now 1786251350122985 us; error 152 us; skew 500 ppm
I20260809 04:55:50.123883 30707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:55:50 dist-test-slave-21b9 krb5kdc[30680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251350, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:55:50.129792 30707 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:55:50.133117 30707 webserver.cc:460] Webserver started at http://127.29.243.254:39363/ using document root <none> and password file <none>
I20260809 04:55:50.134578 30707 fs_manager.cc:359] Metadata directory not provided
I20260809 04:55:50.134805 30707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:55:50.135514 30707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:55:50.141052 30707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "60ee103be638439b8e7673a80f8f328f"
format_stamp: "Formatted at 2026-08-09 04:55:50 on dist-test-slave-21b9"
server_key: "2c01e5e2862297d3c69504357dbfd3d5"
server_key_iv: "95f7224ced7c6076552a397fb9c8abda"
server_key_version: "encryptionkey@0"
I20260809 04:55:50.142079 30707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "60ee103be638439b8e7673a80f8f328f"
format_stamp: "Formatted at 2026-08-09 04:55:50 on dist-test-slave-21b9"
server_key: "2c01e5e2862297d3c69504357dbfd3d5"
server_key_iv: "95f7224ced7c6076552a397fb9c8abda"
server_key_version: "encryptionkey@0"
I20260809 04:55:50.152927 30707 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.011s sys 0.000s
I20260809 04:55:50.160841 30738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:50.163254 30707 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.003s
I20260809 04:55:50.163698 30707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "60ee103be638439b8e7673a80f8f328f"
format_stamp: "Formatted at 2026-08-09 04:55:50 on dist-test-slave-21b9"
server_key: "2c01e5e2862297d3c69504357dbfd3d5"
server_key_iv: "95f7224ced7c6076552a397fb9c8abda"
server_key_version: "encryptionkey@0"
I20260809 04:55:50.164016 30707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:55:50.204408 30707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:55:50.212716 30707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:55:50.213361 30707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:55:50.275628 30707 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:43611
I20260809 04:55:50.275707 30812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:43611 every 8 connection(s)
I20260809 04:55:50.278606 30707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:55:50.284461 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 30707
I20260809 04:55:50.285020 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:55:50.289016 30671 external_mini_cluster.cc:1428] Setting key 062bcfc8ac08bdf9ecbf2e1f5795f9ff
I20260809 04:55:50.290066 30813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:55:50.315013 30813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f: Bootstrap starting.
I20260809 04:55:50.324810 30813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f: Neither blocks nor log segments found. Creating new log.
I20260809 04:55:50.327767 30813 log.cc:824] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:55:50.334039 30813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f: No bootstrap required, opened a new log
Aug 09 04:55:50 dist-test-slave-21b9 krb5kdc[30680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:55:50.349929 30813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ee103be638439b8e7673a80f8f328f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43611 } }
I20260809 04:55:50.350881 30813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:55:50.351236 30813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ee103be638439b8e7673a80f8f328f, State: Initialized, Role: FOLLOWER
I20260809 04:55:50.352702 30813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ee103be638439b8e7673a80f8f328f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43611 } }
I20260809 04:55:50.353412 30813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:55:50.353905 30813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:55:50.354326 30813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:55:50.356251 30816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:50.297482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57478 (local address 127.29.243.254:43611)
0809 04:55:50.299721 (+ 2239us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:50.299741 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:50.303177 (+ 3436us) server_negotiation.cc:407] Connection header received
0809 04:55:50.321149 (+ 17972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:50.321181 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:50.321969 (+ 788us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:50.322538 (+ 569us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:50.327059 (+ 4521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:50.329961 (+ 2902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:50.332776 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:50.334525 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:50.334797 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:50.343141 (+ 8344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:50.343181 (+ 40us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:50.343223 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:50.343336 (+ 113us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:50.348464 (+ 5128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:50.349746 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:50.349754 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:50.349760 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:50.350044 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:50.350831 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:50.350835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:50.350837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:50.351908 (+ 1071us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:50.352357 (+ 449us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:50.353689 (+ 1332us) server_negotiation.cc:299] Negotiation successful
0809 04:55:50.354037 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1746,"thread_start_us":1212,"threads_started":1}
I20260809 04:55:50.358750 30813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ee103be638439b8e7673a80f8f328f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43611 } }
I20260809 04:55:50.359932 30813 leader_election.cc:302] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60ee103be638439b8e7673a80f8f328f; no voters:
I20260809 04:55:50.361949 30813 leader_election.cc:288] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:55:50.362392 30819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:55:50.366139 30813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:55:50.367136 30819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [term 1 LEADER]: Becoming Leader. State: Replica: 60ee103be638439b8e7673a80f8f328f, State: Running, Role: LEADER
I20260809 04:55:50.368546 30819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ee103be638439b8e7673a80f8f328f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43611 } }
I20260809 04:55:50.377511 30820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60ee103be638439b8e7673a80f8f328f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ee103be638439b8e7673a80f8f328f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43611 } } }
I20260809 04:55:50.377808 30822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60ee103be638439b8e7673a80f8f328f. Latest consensus state: current_term: 1 leader_uuid: "60ee103be638439b8e7673a80f8f328f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ee103be638439b8e7673a80f8f328f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43611 } } }
I20260809 04:55:50.379104 30820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [sys.catalog]: This master's current role is: LEADER
I20260809 04:55:50.379213 30822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f [sys.catalog]: This master's current role is: LEADER
I20260809 04:55:50.387065 30835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:55:50.398790 30835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:55:50.422158 30835 catalog_manager.cc:1347] Generated new cluster ID: e65d4cf1befc4937b81e59dfe1751a23
I20260809 04:55:50.422380 30835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:55:50.444123 30835 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:55:50.446558 30835 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:55:50.460539 30835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60ee103be638439b8e7673a80f8f328f: Generated new TSK 0
I20260809 04:55:50.461717 30835 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:55:50.524022 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43611
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:55:51.269547 30855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:55:51.269861 30855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:55:51.270051 30855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:55:51.281407 30855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:55:51.281646 30855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:55:51.281939 30855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:55:51.296725 30855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43611
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:55:51.298277 30855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:55:51.300024 30855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:55:51.319726 30863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:51.319913 30868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:51.320709 30872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:55:51.324930 30855 server_base.cc:1034] running on GCE node
I20260809 04:55:51.325763 30855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:55:51.327021 30855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:55:51.328298 30855 hybrid_clock.cc:648] HybridClock initialized: now 1786251351328190 us; error 31 us; skew 500 ppm
I20260809 04:55:51.328743 30855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:55:51 dist-test-slave-21b9 krb5kdc[30680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251351, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:55:51.333798 30855 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:55:51.336449 30855 webserver.cc:460] Webserver started at http://127.29.243.193:38205/ using document root <none> and password file <none>
I20260809 04:55:51.337435 30855 fs_manager.cc:359] Metadata directory not provided
I20260809 04:55:51.337656 30855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:55:51.338135 30855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:55:51.342314 30855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "101783a639bf4b1391c22374aa22242b"
format_stamp: "Formatted at 2026-08-09 04:55:51 on dist-test-slave-21b9"
server_key: "a594a9124045b5d33b3803a65f8ff0f3"
server_key_iv: "cfaa0b1de3f5eafa89f7fcfa9c068f1d"
server_key_version: "encryptionkey@0"
I20260809 04:55:51.343235 30855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "101783a639bf4b1391c22374aa22242b"
format_stamp: "Formatted at 2026-08-09 04:55:51 on dist-test-slave-21b9"
server_key: "a594a9124045b5d33b3803a65f8ff0f3"
server_key_iv: "cfaa0b1de3f5eafa89f7fcfa9c068f1d"
server_key_version: "encryptionkey@0"
I20260809 04:55:51.350688 30855 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260809 04:55:51.356112 30885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:51.357707 30855 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:55:51.357951 30855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "101783a639bf4b1391c22374aa22242b"
format_stamp: "Formatted at 2026-08-09 04:55:51 on dist-test-slave-21b9"
server_key: "a594a9124045b5d33b3803a65f8ff0f3"
server_key_iv: "cfaa0b1de3f5eafa89f7fcfa9c068f1d"
server_key_version: "encryptionkey@0"
I20260809 04:55:51.358228 30855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:55:51.391083 30855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:55:51.397382 30855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:55:51.397897 30855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:55:51.399725 30855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:55:51.402491 30855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:55:51.402624 30855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:51.402928 30855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:55:51.403064 30855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:51.499516 30855 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:42377
I20260809 04:55:51.499791 31073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:42377 every 8 connection(s)
I20260809 04:55:51.501386 30855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:55:51.503400 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 30855
I20260809 04:55:51.503820 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:55:51.504755 30671 external_mini_cluster.cc:1428] Setting key 8fbe83386a6f9ff91112298c75a5dad9
Aug 09 04:55:51 dist-test-slave-21b9 krb5kdc[30680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251351, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
I20260809 04:55:51.533870 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:51.506523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:34025 (local address 127.29.243.254:43611)
0809 04:55:51.508003 (+ 1480us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:51.508011 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:51.509823 (+ 1812us) server_negotiation.cc:407] Connection header received
0809 04:55:51.513153 (+ 3330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:51.513159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:51.513247 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:51.513372 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:51.515731 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:51.517231 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:51.519589 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:51.520309 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:51.520434 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:51.525686 (+ 5252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:51.525694 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:51.525697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:51.525732 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:51.530149 (+ 4417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:51.531007 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:51.531011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:51.531014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:51.531207 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:51.531746 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:51.531750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:51.531752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:51.532112 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:51.532331 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:51.533119 (+ 788us) server_negotiation.cc:299] Negotiation successful
0809 04:55:51.533335 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1299,"thread_start_us":1131,"threads_started":1}
I20260809 04:55:51.535192 31078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:51.507293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:43611 (local address 127.29.243.193:34025)
0809 04:55:51.509120 (+ 1827us) negotiation.cc:107] Waiting for socket to connect
0809 04:55:51.509177 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:55:51.512857 (+ 3680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:55:51.513623 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:55:51.513632 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:55:51.514502 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:55:51.515518 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:51.515542 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:51.517393 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:51.517397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:51.519432 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:51.519455 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:51.520679 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:51.520688 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:51.520980 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:51.521782 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:55:51.521806 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:55:51.525324 (+ 3518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:55:51.530287 (+ 4963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:51.530296 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:51.530312 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:51.530880 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:51.531357 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:51.531370 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:51.531372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:51.531624 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:51.532288 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:55:51.532297 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:55:51.532723 (+ 426us) client_negotiation.cc:769] Sending connection context
0809 04:55:51.533006 (+ 283us) client_negotiation.cc:240] Negotiation successful
0809 04:55:51.533640 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1389,"thread_start_us":932,"threads_started":1}
I20260809 04:55:51.536942 31074 heartbeater.cc:347] Connected to a master server at 127.29.243.254:43611
I20260809 04:55:51.537475 31074 heartbeater.cc:464] Registering TS with master...
I20260809 04:55:51.538625 31074 heartbeater.cc:511] Master 127.29.243.254:43611 requested a full tablet report, sending...
I20260809 04:55:51.542876 30762 ts_manager.cc:194] Registered new tserver with Master: 101783a639bf4b1391c22374aa22242b (127.29.243.193:42377)
I20260809 04:55:51.545674 30762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:34025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:55:51.552668 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43611
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:55:52.285815 31082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:55:52.286381 31082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:55:52.286790 31082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:55:52.297711 31082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:55:52.297921 31082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:55:52.298130 31082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:55:52.312613 31082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43611
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:55:52.314165 31082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:55:52.315922 31082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:55:52.336403 31094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:52.336879 31095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:52.337352 31100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:55:52.338471 31082 server_base.cc:1034] running on GCE node
I20260809 04:55:52.339202 31082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:55:52.340477 31082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:55:52.341774 31082 hybrid_clock.cc:648] HybridClock initialized: now 1786251352341655 us; error 45 us; skew 500 ppm
I20260809 04:55:52.342240 31082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:55:52 dist-test-slave-21b9 krb5kdc[30680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251352, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:55:52.347434 31082 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:55:52.349534 31082 webserver.cc:460] Webserver started at http://127.29.243.194:41731/ using document root <none> and password file <none>
I20260809 04:55:52.350489 31082 fs_manager.cc:359] Metadata directory not provided
I20260809 04:55:52.350701 31082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:55:52.351145 31082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:55:52.355533 31082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "ecb4fd9a9a0845e5b76995501771271f"
format_stamp: "Formatted at 2026-08-09 04:55:52 on dist-test-slave-21b9"
server_key: "7bad232ac722199b476544112ad68e39"
server_key_iv: "31361362d9c33333be17ee74151f1a84"
server_key_version: "encryptionkey@0"
I20260809 04:55:52.356498 31082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "ecb4fd9a9a0845e5b76995501771271f"
format_stamp: "Formatted at 2026-08-09 04:55:52 on dist-test-slave-21b9"
server_key: "7bad232ac722199b476544112ad68e39"
server_key_iv: "31361362d9c33333be17ee74151f1a84"
server_key_version: "encryptionkey@0"
I20260809 04:55:52.363690 31082 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.002s
I20260809 04:55:52.369580 31120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:52.371090 31082 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:55:52.371418 31082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "ecb4fd9a9a0845e5b76995501771271f"
format_stamp: "Formatted at 2026-08-09 04:55:52 on dist-test-slave-21b9"
server_key: "7bad232ac722199b476544112ad68e39"
server_key_iv: "31361362d9c33333be17ee74151f1a84"
server_key_version: "encryptionkey@0"
I20260809 04:55:52.371827 31082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:55:52.390930 31082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:55:52.397684 31082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:55:52.398216 31082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:55:52.400209 31082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:55:52.402549 31082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:55:52.402686 31082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:52.402927 31082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:55:52.403057 31082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:52.496732 31082 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:36145
I20260809 04:55:52.496994 31304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:36145 every 8 connection(s)
I20260809 04:55:52.498564 31082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:55:52.499590 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 31082
I20260809 04:55:52.499930 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:55:52.500739 30671 external_mini_cluster.cc:1428] Setting key 51870900ed0833b16d4f6e3b00fca413
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Aug 09 04:55:52 dist-test-slave-21b9 krb5kdc[30680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251352, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:55:52.535398 31309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:52.505425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:47317 (local address 127.29.243.254:43611)
0809 04:55:52.507369 (+ 1944us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:52.507377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:52.512997 (+ 5620us) server_negotiation.cc:407] Connection header received
0809 04:55:52.516126 (+ 3129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:52.516131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:52.516275 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:52.516376 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:52.518678 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:52.519840 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:52.522383 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:52.523140 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:52.523259 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:52.528843 (+ 5584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:52.528850 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:52.528854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:52.528913 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:52.531772 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:52.532728 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:52.532731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:52.532734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:52.532847 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:52.533493 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:52.533496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:52.533498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:52.533763 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:52.533872 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:52.534898 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0809 04:55:52.535091 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1744,"thread_start_us":6333,"threads_started":1}
I20260809 04:55:52.537281 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:52.503637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:43611 (local address 127.29.243.194:47317)
0809 04:55:52.512823 (+ 9186us) negotiation.cc:107] Waiting for socket to connect
0809 04:55:52.512874 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 04:55:52.515841 (+ 2967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:55:52.516679 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:55:52.516689 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:55:52.517491 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:55:52.518467 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:52.518521 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:52.520089 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:52.520093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:52.522222 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:52.522236 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:52.523423 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:52.523433 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:52.523693 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:52.524569 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:55:52.524595 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:55:52.528585 (+ 3990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:55:52.531911 (+ 3326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:52.531920 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:52.531937 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:52.532629 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:52.532939 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:52.532944 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:52.532948 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:52.533385 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:52.533886 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:55:52.533894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:55:52.534453 (+ 559us) client_negotiation.cc:769] Sending connection context
0809 04:55:52.534817 (+ 364us) client_negotiation.cc:240] Negotiation successful
0809 04:55:52.535472 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8778,"thread_start_us":8279,"threads_started":1}
I20260809 04:55:52.539028 31305 heartbeater.cc:347] Connected to a master server at 127.29.243.254:43611
I20260809 04:55:52.539434 31305 heartbeater.cc:464] Registering TS with master...
I20260809 04:55:52.540637 31305 heartbeater.cc:511] Master 127.29.243.254:43611 requested a full tablet report, sending...
I20260809 04:55:52.542415 30762 ts_manager.cc:194] Registered new tserver with Master: ecb4fd9a9a0845e5b76995501771271f (127.29.243.194:36145)
I20260809 04:55:52.543828 30762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:47317
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:55:52.550261 31074 heartbeater.cc:503] Master 127.29.243.254:43611 was elected leader, sending a full tablet report...
I20260809 04:55:52.550896 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43611
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:55:53.282043 31317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:55:53.282438 31317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:55:53.282660 31317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:55:53.294253 31317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:55:53.294505 31317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:55:53.294761 31317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:55:53.308825 31317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43611
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:55:53.310587 31317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:55:53.312558 31317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:55:53.332545 31326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:53.333314 31327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:53.333573 31332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:55:53.334626 31317 server_base.cc:1034] running on GCE node
I20260809 04:55:53.335430 31317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:55:53.336659 31317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:55:53.337941 31317 hybrid_clock.cc:648] HybridClock initialized: now 1786251353337827 us; error 38 us; skew 500 ppm
I20260809 04:55:53.338411 31317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:55:53 dist-test-slave-21b9 krb5kdc[30680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251353, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:55:53.347702 31317 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:55:53.349965 31317 webserver.cc:460] Webserver started at http://127.29.243.195:42645/ using document root <none> and password file <none>
I20260809 04:55:53.351100 31317 fs_manager.cc:359] Metadata directory not provided
I20260809 04:55:53.351358 31317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:55:53.351855 31317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:55:53.356012 31317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "72c8ac311b0341b3a813b2c85b144dcf"
format_stamp: "Formatted at 2026-08-09 04:55:53 on dist-test-slave-21b9"
server_key: "f47b30fbd4cc441778921b9a8201e5ef"
server_key_iv: "63d5d90a522a5f94807f7ae70defb8e4"
server_key_version: "encryptionkey@0"
I20260809 04:55:53.356978 31317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "72c8ac311b0341b3a813b2c85b144dcf"
format_stamp: "Formatted at 2026-08-09 04:55:53 on dist-test-slave-21b9"
server_key: "f47b30fbd4cc441778921b9a8201e5ef"
server_key_iv: "63d5d90a522a5f94807f7ae70defb8e4"
server_key_version: "encryptionkey@0"
I20260809 04:55:53.364303 31317 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260809 04:55:53.370039 31341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:53.371650 31317 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260809 04:55:53.371794 31317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "72c8ac311b0341b3a813b2c85b144dcf"
format_stamp: "Formatted at 2026-08-09 04:55:53 on dist-test-slave-21b9"
server_key: "f47b30fbd4cc441778921b9a8201e5ef"
server_key_iv: "63d5d90a522a5f94807f7ae70defb8e4"
server_key_version: "encryptionkey@0"
I20260809 04:55:53.371971 31317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:55:53.409590 31317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:55:53.415876 31317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:55:53.416363 31317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:55:53.418357 31317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:55:53.420831 31317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:55:53.420977 31317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:53.421241 31317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:55:53.421377 31317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:53.521929 31317 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:41131
I20260809 04:55:53.522253 31515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:41131 every 8 connection(s)
I20260809 04:55:53.523914 31317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:55:53.525651 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 31317
I20260809 04:55:53.526028 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:55:53.526958 30671 external_mini_cluster.cc:1428] Setting key de511ad1fee66e3d52b831b0a82bcfc5
Aug 09 04:55:53 dist-test-slave-21b9 krb5kdc[30680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251353, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:55:53.546860 31305 heartbeater.cc:503] Master 127.29.243.254:43611 was elected leader, sending a full tablet report...
I20260809 04:55:53.555013 31520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:53.528227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:49637 (local address 127.29.243.254:43611)
0809 04:55:53.529142 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:53.529147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:53.531261 (+ 2114us) server_negotiation.cc:407] Connection header received
0809 04:55:53.534693 (+ 3432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:53.534698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:53.534764 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:53.534851 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:53.537044 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.538254 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.540615 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.541280 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.541385 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:53.546412 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:53.546418 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:53.546421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:53.546455 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:53.550820 (+ 4365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.551880 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.551885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.551887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.552118 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.552856 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.552871 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.552873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.553323 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:53.553465 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:53.554229 (+ 764us) server_negotiation.cc:299] Negotiation successful
0809 04:55:53.554442 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":560,"threads_started":1}
I20260809 04:55:53.556145 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:53.529384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:43611 (local address 127.29.243.195:49637)
0809 04:55:53.531081 (+ 1697us) negotiation.cc:107] Waiting for socket to connect
0809 04:55:53.531142 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 04:55:53.534408 (+ 3266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:55:53.535112 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:55:53.535120 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:55:53.536020 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:55:53.536896 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:53.536917 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.538446 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.538450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:53.540490 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:53.540505 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.541510 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.541517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:53.541771 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:53.542466 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:55:53.542490 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:55:53.546131 (+ 3641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:55:53.551089 (+ 4958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:53.551098 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:53.551113 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:53.551769 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:53.552250 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:53.552254 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:53.552257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:53.552713 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:53.553459 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:55:53.553467 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:55:53.553883 (+ 416us) client_negotiation.cc:769] Sending connection context
0809 04:55:53.554141 (+ 258us) client_negotiation.cc:240] Negotiation successful
0809 04:55:53.554675 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1325,"thread_start_us":909,"threads_started":1}
I20260809 04:55:53.557839 31516 heartbeater.cc:347] Connected to a master server at 127.29.243.254:43611
I20260809 04:55:53.558218 31516 heartbeater.cc:464] Registering TS with master...
I20260809 04:55:53.559391 31516 heartbeater.cc:511] Master 127.29.243.254:43611 requested a full tablet report, sending...
I20260809 04:55:53.561043 30762 ts_manager.cc:194] Registered new tserver with Master: 72c8ac311b0341b3a813b2c85b144dcf (127.29.243.195:41131)
I20260809 04:55:53.562157 30762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:49637
I20260809 04:55:53.570969 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:55:53 dist-test-slave-21b9 krb5kdc[30680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251353, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:55:53 dist-test-slave-21b9 krb5kdc[30680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251353, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:55:53.631824 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:53.607546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35222 (local address 127.29.243.193:42377)
0809 04:55:53.608872 (+ 1326us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:53.608884 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:53.609659 (+ 775us) server_negotiation.cc:407] Connection header received
0809 04:55:53.609927 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:53.609937 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:53.610178 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:53.610406 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:53.613235 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.614762 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.616816 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.617753 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.617999 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:53.623941 (+ 5942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:53.623956 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:53.623973 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:53.624014 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:53.628216 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.629052 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.629060 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.629074 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.629264 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.629739 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.629742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.629744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.630208 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:53.630610 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:53.630929 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 04:55:53.631212 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1063,"thread_start_us":740,"threads_started":1}
W20260809 04:55:53.633369 30932 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35222
Aug 09 04:55:53 dist-test-slave-21b9 krb5kdc[30680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:55:53 dist-test-slave-21b9 krb5kdc[30680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:55:53.690848 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:53.671018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35226 (local address 127.29.243.193:42377)
0809 04:55:53.671210 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:53.671217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:53.671835 (+ 618us) server_negotiation.cc:407] Connection header received
0809 04:55:53.672019 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:53.672024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:53.672185 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:53.672281 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:53.674976 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.676059 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.677934 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.678689 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.678804 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:53.684975 (+ 6171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:53.684980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:53.684983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:53.685018 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:53.688131 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.688810 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.688814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.688816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.688951 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.689368 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.689372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.689374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.689654 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:53.689884 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:53.690229 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 04:55:53.690468 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:55:53.692675 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 30855
I20260809 04:55:53.709470 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:53.693760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35228 (local address 127.29.243.193:42377)
0809 04:55:53.693912 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:53.693916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:53.694662 (+ 746us) server_negotiation.cc:407] Connection header received
0809 04:55:53.694823 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:53.694827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:53.694884 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:53.694963 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:53.696806 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.697591 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.699233 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.699757 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.699871 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:53.702843 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:53.702848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:53.702851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:53.702880 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:53.706264 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.707022 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.707026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.707028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.707229 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.707666 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.707670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.707672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.708033 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:53.708392 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:53.708719 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 04:55:53.708961 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:55:53.710585 31065 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:55:53.800963 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 30855
I20260809 04:55:53.825412 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31082
Aug 09 04:55:53 dist-test-slave-21b9 krb5kdc[30680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:55:53.843711 31541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:53.826332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.29.243.194:36145)
0809 04:55:53.827194 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:53.827202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:53.827229 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:55:53.827324 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:53.827333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:53.827596 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:53.827771 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:53.829373 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.830916 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.832667 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.833638 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.833736 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:53.838081 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:53.838093 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:53.838109 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:53.838156 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:53.841170 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.841742 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.841749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.841751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.841885 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.842250 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.842255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.842257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.842641 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:53.842881 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:53.843085 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 04:55:53.843332 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":386,"threads_started":1}
I20260809 04:55:53.845000 31299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:55:53.928534 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 31082
I20260809 04:55:53.953406 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31317
Aug 09 04:55:53 dist-test-slave-21b9 krb5kdc[30680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:55:53.970377 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:53.954422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.29.243.195:41131)
0809 04:55:53.955534 (+ 1112us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:53.955555 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:53.955611 (+ 56us) server_negotiation.cc:407] Connection header received
0809 04:55:53.955706 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:53.955714 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:53.955931 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:53.956094 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:53.957214 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.958675 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.960034 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:53.960583 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:53.960677 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:53.964635 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:53.964647 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:53.964663 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:53.964698 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:53.967612 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.968240 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.968248 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.968250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.968399 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:53.968751 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:53.968756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:53.968760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:53.969139 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:53.969454 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:53.969765 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 04:55:53.969968 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":917,"thread_start_us":515,"threads_started":1}
I20260809 04:55:53.971740 31510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:55:54.056427 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 31317
I20260809 04:55:54.082957 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 30707
I20260809 04:55:54.084239 30806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:55:54.186960 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 30707
2026-08-09T04:55:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (6306 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:55:54 dist-test-slave-21b9 krb5kdc[31553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:55:54 dist-test-slave-21b9 krb5kdc[31553](info): set up 2 sockets
Aug 09 04:55:54 dist-test-slave-21b9 krb5kdc[31553](info): commencing operation
krb5kdc: starting...
W20260809 04:55:55.391649 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:55:55 dist-test-slave-21b9 krb5kdc[31553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:55:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:55:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:55:55.590487 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44667
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:44667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:55:55.928670 31578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:55:55.928956 31578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:55:55.929051 31578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:55:55.936354 31578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:55:55.936468 31578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:55:55.936523 31578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:55:55.936592 31578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:55:55.936636 31578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:55:55.946475 31578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:44667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:55:55.947856 31578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:55:55.949227 31578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:55:55.960775 31589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:55.960824 31590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:55.960850 31593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:55:55.961954 31578 server_base.cc:1034] running on GCE node
I20260809 04:55:55.962666 31578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:55:55.964048 31578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:55:55.965255 31578 hybrid_clock.cc:648] HybridClock initialized: now 1786251355965194 us; error 45 us; skew 500 ppm
I20260809 04:55:55.965605 31578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:55:55 dist-test-slave-21b9 krb5kdc[31553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251355, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:55:55.968644 31578 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:55:55.970105 31578 webserver.cc:460] Webserver started at http://127.29.243.254:35643/ using document root <none> and password file <none>
I20260809 04:55:55.970775 31578 fs_manager.cc:359] Metadata directory not provided
I20260809 04:55:55.970880 31578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:55:55.971158 31578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:55:55.974076 31578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "83acec95d6624c159c01274fba799b79"
format_stamp: "Formatted at 2026-08-09 04:55:55 on dist-test-slave-21b9"
server_key: "c5360659334a3c5b05d9be9161b4383f"
server_key_iv: "255732dafc0845683e00707afc8a1526"
server_key_version: "encryptionkey@0"
I20260809 04:55:55.974768 31578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "83acec95d6624c159c01274fba799b79"
format_stamp: "Formatted at 2026-08-09 04:55:55 on dist-test-slave-21b9"
server_key: "c5360659334a3c5b05d9be9161b4383f"
server_key_iv: "255732dafc0845683e00707afc8a1526"
server_key_version: "encryptionkey@0"
I20260809 04:55:55.979880 31578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 04:55:55.983289 31602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:55.984634 31578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:55:55.984781 31578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "83acec95d6624c159c01274fba799b79"
format_stamp: "Formatted at 2026-08-09 04:55:55 on dist-test-slave-21b9"
server_key: "c5360659334a3c5b05d9be9161b4383f"
server_key_iv: "255732dafc0845683e00707afc8a1526"
server_key_version: "encryptionkey@0"
I20260809 04:55:55.984941 31578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:55:56.017119 31578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:55:56.023031 31578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:55:56.023420 31578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:55:56.042232 31578 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:44667
I20260809 04:55:56.042240 31683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:44667 every 8 connection(s)
I20260809 04:55:56.043736 31578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:55:56.047731 31685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:55:56.049091 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 31578
I20260809 04:55:56.049301 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:55:56.049944 30671 external_mini_cluster.cc:1428] Setting key ef1c2c73196016712ff394bb4b9e1215
I20260809 04:55:56.062477 31685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79: Bootstrap starting.
Aug 09 04:55:56 dist-test-slave-21b9 krb5kdc[31553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:55:56.066850 31685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79: Neither blocks nor log segments found. Creating new log.
I20260809 04:55:56.068063 31685 log.cc:824] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79: Log is configured to *not* fsync() on all Append() calls
I20260809 04:55:56.071631 31685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79: No bootstrap required, opened a new log
I20260809 04:55:56.074373 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:56.051987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36064 (local address 127.29.243.254:44667)
0809 04:55:56.052848 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:56.052860 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:56.052906 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:55:56.054538 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:56.054563 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:56.054959 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:56.055318 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:56.057381 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:56.058730 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:56.060625 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:56.061317 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:56.061444 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:56.066589 (+ 5145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:56.066597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:56.066613 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:56.066651 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:56.070026 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:56.070906 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:56.070918 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:56.070925 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:56.071105 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:56.071559 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:56.071562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:56.071564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:56.072146 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:56.072377 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:56.072937 (+ 560us) server_negotiation.cc:299] Negotiation successful
0809 04:55:56.073275 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":352,"threads_started":1}
I20260809 04:55:56.080197 31685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83acec95d6624c159c01274fba799b79" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44667 } }
I20260809 04:55:56.080586 31685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:55:56.080687 31685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83acec95d6624c159c01274fba799b79, State: Initialized, Role: FOLLOWER
I20260809 04:55:56.081375 31685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83acec95d6624c159c01274fba799b79" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44667 } }
I20260809 04:55:56.081681 31685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:55:56.081826 31685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:55:56.082005 31685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:55:56.084545 31685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83acec95d6624c159c01274fba799b79" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44667 } }
I20260809 04:55:56.085120 31685 leader_election.cc:302] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83acec95d6624c159c01274fba799b79; no voters:
I20260809 04:55:56.085721 31685 leader_election.cc:288] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:55:56.085829 31695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:55:56.086490 31695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [term 1 LEADER]: Becoming Leader. State: Replica: 83acec95d6624c159c01274fba799b79, State: Running, Role: LEADER
I20260809 04:55:56.087009 31695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83acec95d6624c159c01274fba799b79" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44667 } }
I20260809 04:55:56.087975 31685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:55:56.089763 31697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83acec95d6624c159c01274fba799b79. Latest consensus state: current_term: 1 leader_uuid: "83acec95d6624c159c01274fba799b79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83acec95d6624c159c01274fba799b79" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44667 } } }
I20260809 04:55:56.089776 31696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83acec95d6624c159c01274fba799b79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83acec95d6624c159c01274fba799b79" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44667 } } }
I20260809 04:55:56.090237 31696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [sys.catalog]: This master's current role is: LEADER
I20260809 04:55:56.090237 31697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79 [sys.catalog]: This master's current role is: LEADER
I20260809 04:55:56.091146 31702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:55:56.096130 31702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:55:56.105326 31702 catalog_manager.cc:1347] Generated new cluster ID: f0f4a90c58234e0bae15d8b3a78c12a4
I20260809 04:55:56.105477 31702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:55:56.130407 31702 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:55:56.131783 31702 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:55:56.143077 31702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83acec95d6624c159c01274fba799b79: Generated new TSK 0
I20260809 04:55:56.143962 31702 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:55:56.193164 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44667
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:55:56.528950 31729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:55:56.529248 31729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:55:56.529346 31729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:55:56.536473 31729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:55:56.536580 31729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:55:56.536693 31729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:55:56.546195 31729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44667
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:55:56.547452 31729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:55:56.548745 31729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:55:56.561522 31738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:56.561565 31743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:56.561600 31740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:55:56.562608 31729 server_base.cc:1034] running on GCE node
I20260809 04:55:56.563161 31729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:55:56.564138 31729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:55:56.565353 31729 hybrid_clock.cc:648] HybridClock initialized: now 1786251356565300 us; error 46 us; skew 500 ppm
I20260809 04:55:56.565677 31729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:55:56 dist-test-slave-21b9 krb5kdc[31553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251356, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:55:56.568792 31729 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:55:56.570272 31729 webserver.cc:460] Webserver started at http://127.29.243.193:42151/ using document root <none> and password file <none>
I20260809 04:55:56.570946 31729 fs_manager.cc:359] Metadata directory not provided
I20260809 04:55:56.571053 31729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:55:56.571372 31729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:55:56.574277 31729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "eb124a8217924048973b2c065b72d0e5"
format_stamp: "Formatted at 2026-08-09 04:55:56 on dist-test-slave-21b9"
server_key: "8d07d86ccad7563e2a9301a32aa2637f"
server_key_iv: "a8fd1afa04678f26c42626111156fd77"
server_key_version: "encryptionkey@0"
I20260809 04:55:56.575063 31729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "eb124a8217924048973b2c065b72d0e5"
format_stamp: "Formatted at 2026-08-09 04:55:56 on dist-test-slave-21b9"
server_key: "8d07d86ccad7563e2a9301a32aa2637f"
server_key_iv: "a8fd1afa04678f26c42626111156fd77"
server_key_version: "encryptionkey@0"
I20260809 04:55:56.580353 31729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 04:55:56.584599 31752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:56.586340 31729 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:55:56.586514 31729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "eb124a8217924048973b2c065b72d0e5"
format_stamp: "Formatted at 2026-08-09 04:55:56 on dist-test-slave-21b9"
server_key: "8d07d86ccad7563e2a9301a32aa2637f"
server_key_iv: "a8fd1afa04678f26c42626111156fd77"
server_key_version: "encryptionkey@0"
I20260809 04:55:56.586750 31729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:55:56.606685 31729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:55:56.610854 31729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:55:56.611109 31729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:55:56.612074 31729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:55:56.613354 31729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:55:56.613430 31729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:56.613524 31729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:55:56.613566 31729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:56.644496 31729 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:41279
I20260809 04:55:56.644583 31933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:41279 every 8 connection(s)
I20260809 04:55:56.645788 31729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:55:56.652133 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 31729
I20260809 04:55:56.652335 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:55:56.652938 30671 external_mini_cluster.cc:1428] Setting key a72df246e0fd7c1400b92b8900884955
Aug 09 04:55:56 dist-test-slave-21b9 krb5kdc[31553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251356, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:55:56.665956 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:56.648346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:44141 (local address 127.29.243.254:44667)
0809 04:55:56.648780 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:56.648784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:56.649689 (+ 905us) server_negotiation.cc:407] Connection header received
0809 04:55:56.651548 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:56.651553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:56.651644 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:56.651740 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:56.653354 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:56.654222 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:56.655948 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:56.656350 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:56.656463 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:56.659901 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:56.659907 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:56.659910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:56.659943 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:56.662722 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:56.663513 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:56.663519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:56.663522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:56.663670 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:56.664083 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:56.664088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:56.664091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:56.664452 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:56.664595 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:56.665397 (+ 802us) server_negotiation.cc:299] Negotiation successful
0809 04:55:56.665629 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":207,"threads_started":1}
I20260809 04:55:56.666775 31938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:56.648811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44667 (local address 127.29.243.193:44141)
0809 04:55:56.649526 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 04:55:56.649564 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 04:55:56.651233 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:55:56.651993 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:55:56.652001 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:55:56.652596 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:55:56.653189 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:56.653209 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:56.654405 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:56.654409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:56.655807 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:56.655821 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:56.656577 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:56.656584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:56.656805 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:56.657484 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:55:56.657508 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:55:56.659683 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:55:56.662880 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:56.662887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:56.662901 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:56.663382 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:56.663769 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:56.663773 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:56.663775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:56.663971 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:56.664605 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:55:56.664612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:55:56.664942 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 04:55:56.665212 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 04:55:56.665623 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":271,"threads_started":1}
I20260809 04:55:56.668308 31934 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44667
I20260809 04:55:56.668618 31934 heartbeater.cc:464] Registering TS with master...
I20260809 04:55:56.669435 31934 heartbeater.cc:511] Master 127.29.243.254:44667 requested a full tablet report, sending...
I20260809 04:55:56.671696 31628 ts_manager.cc:194] Registered new tserver with Master: eb124a8217924048973b2c065b72d0e5 (127.29.243.193:41279)
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
I20260809 04:55:56.673214 31628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:44141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:55:56.698588 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44667
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:55:57.026384 31944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:55:57.026682 31944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:55:57.026818 31944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:55:57.034125 31944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:55:57.034265 31944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:55:57.034430 31944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:55:57.044407 31944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44667
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:55:57.045748 31944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:55:57.047097 31944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:55:57.060750 31951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:57.060781 31952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:57.060808 31959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:55:57.061695 31944 server_base.cc:1034] running on GCE node
I20260809 04:55:57.062346 31944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:55:57.063345 31944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:55:57.064584 31944 hybrid_clock.cc:648] HybridClock initialized: now 1786251357064533 us; error 37 us; skew 500 ppm
I20260809 04:55:57.064918 31944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:55:57 dist-test-slave-21b9 krb5kdc[31553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251357, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:55:57.068120 31944 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:55:57.069679 31944 webserver.cc:460] Webserver started at http://127.29.243.194:44921/ using document root <none> and password file <none>
I20260809 04:55:57.070425 31944 fs_manager.cc:359] Metadata directory not provided
I20260809 04:55:57.070535 31944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:55:57.070823 31944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:55:57.073952 31944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e"
format_stamp: "Formatted at 2026-08-09 04:55:57 on dist-test-slave-21b9"
server_key: "85cdc6cc0284932770dd9ad6a0ce8486"
server_key_iv: "7c7ea82014e243b843e1a075c212195a"
server_key_version: "encryptionkey@0"
I20260809 04:55:57.074759 31944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e"
format_stamp: "Formatted at 2026-08-09 04:55:57 on dist-test-slave-21b9"
server_key: "85cdc6cc0284932770dd9ad6a0ce8486"
server_key_iv: "7c7ea82014e243b843e1a075c212195a"
server_key_version: "encryptionkey@0"
I20260809 04:55:57.080267 31944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 04:55:57.083981 31971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:57.085397 31944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:55:57.085556 31944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e"
format_stamp: "Formatted at 2026-08-09 04:55:57 on dist-test-slave-21b9"
server_key: "85cdc6cc0284932770dd9ad6a0ce8486"
server_key_iv: "7c7ea82014e243b843e1a075c212195a"
server_key_version: "encryptionkey@0"
I20260809 04:55:57.085749 31944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:55:57.126010 31944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:55:57.130694 31944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:55:57.130997 31944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:55:57.132040 31944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:55:57.133522 31944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:55:57.133605 31944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:57.133731 31944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:55:57.133787 31944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:57.163817 31944 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:35849
I20260809 04:55:57.163887 32145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:35849 every 8 connection(s)
I20260809 04:55:57.165138 31944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:55:57.167656 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 31944
I20260809 04:55:57.167882 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:55:57.168404 30671 external_mini_cluster.cc:1428] Setting key afe7ece628aeb90d5af7b0fc8ae4aeac
Aug 09 04:55:57 dist-test-slave-21b9 krb5kdc[31553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251357, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
I20260809 04:55:57.191699 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:57.167834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:41915 (local address 127.29.243.254:44667)
0809 04:55:57.175625 (+ 7791us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:57.175629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:57.175645 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:55:57.175694 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:57.175697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:57.175764 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:57.175851 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:57.177753 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.178820 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.180487 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.180916 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.181040 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:57.185200 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:57.185206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:57.185210 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:57.185250 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:57.188350 (+ 3100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:57.189198 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:57.189205 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:57.189209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:57.189384 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:57.189829 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:57.189835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:57.189838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:57.190113 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:57.190288 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:57.191044 (+ 756us) server_negotiation.cc:299] Negotiation successful
0809 04:55:57.191307 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7686,"thread_start_us":7584,"threads_started":1}
I20260809 04:55:57.192783 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:57.168516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44667 (local address 127.29.243.194:41915)
0809 04:55:57.169468 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0809 04:55:57.169524 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 04:55:57.172124 (+ 2600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:55:57.176170 (+ 4046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:55:57.176182 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:55:57.176915 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:55:57.177544 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:57.177564 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.178986 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.178990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:57.180327 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:57.180343 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.181152 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.181160 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:57.181392 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:57.182153 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:55:57.182179 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:55:57.184886 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:55:57.188489 (+ 3603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:57.188497 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:57.188527 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:57.189058 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:57.189496 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:57.189501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:57.189504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:57.189707 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:57.190263 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:55:57.190271 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:55:57.190660 (+ 389us) client_negotiation.cc:769] Sending connection context
0809 04:55:57.190927 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 04:55:57.191509 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":348,"threads_started":1}
I20260809 04:55:57.194387 32146 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44667
I20260809 04:55:57.194718 32146 heartbeater.cc:464] Registering TS with master...
I20260809 04:55:57.195531 32146 heartbeater.cc:511] Master 127.29.243.254:44667 requested a full tablet report, sending...
I20260809 04:55:57.197017 31628 ts_manager.cc:194] Registered new tserver with Master: ccf0cb4a6e8e445c87ce5a43230bab4e (127.29.243.194:35849)
I20260809 04:55:57.197897 31628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:41915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:55:57.215230 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44667
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:55:57.554435 32158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:55:57.554829 32158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:55:57.555070 32158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:55:57.565665 32158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:55:57.565882 32158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:55:57.566133 32158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:55:57.579901 32158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44667
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:55:57.581439 32158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:55:57.583101 32158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:55:57.602277 32166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:57.603111 32167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:55:57.603221 32170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:55:57.604398 32158 server_base.cc:1034] running on GCE node
I20260809 04:55:57.605046 32158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:55:57.606077 32158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:55:57.607347 32158 hybrid_clock.cc:648] HybridClock initialized: now 1786251357607238 us; error 41 us; skew 500 ppm
I20260809 04:55:57.607802 32158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:55:57 dist-test-slave-21b9 krb5kdc[31553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251357, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:55:57.612438 32158 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:55:57.614724 32158 webserver.cc:460] Webserver started at http://127.29.243.195:35855/ using document root <none> and password file <none>
I20260809 04:55:57.615664 32158 fs_manager.cc:359] Metadata directory not provided
I20260809 04:55:57.615846 32158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:55:57.616255 32158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:55:57.620128 32158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "7ac2ed5251504c7385f1f946e29af4c2"
format_stamp: "Formatted at 2026-08-09 04:55:57 on dist-test-slave-21b9"
server_key: "d3ffca27afba9bf2fc98f68ab45aebd1"
server_key_iv: "e311cbdcc7008ebf897658cf136035a5"
server_key_version: "encryptionkey@0"
I20260809 04:55:57.621064 32158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "7ac2ed5251504c7385f1f946e29af4c2"
format_stamp: "Formatted at 2026-08-09 04:55:57 on dist-test-slave-21b9"
server_key: "d3ffca27afba9bf2fc98f68ab45aebd1"
server_key_iv: "e311cbdcc7008ebf897658cf136035a5"
server_key_version: "encryptionkey@0"
I20260809 04:55:57.627924 32158 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260809 04:55:57.633595 32179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:57.635116 32158 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 04:55:57.635340 32158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "7ac2ed5251504c7385f1f946e29af4c2"
format_stamp: "Formatted at 2026-08-09 04:55:57 on dist-test-slave-21b9"
server_key: "d3ffca27afba9bf2fc98f68ab45aebd1"
server_key_iv: "e311cbdcc7008ebf897658cf136035a5"
server_key_version: "encryptionkey@0"
I20260809 04:55:57.635605 32158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:55:57.655467 32158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:55:57.661600 32158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:55:57.662063 32158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:55:57.663905 32158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:55:57.666287 32158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:55:57.666447 32158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:57.666651 32158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:55:57.666798 32158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:55:57.676399 31934 heartbeater.cc:503] Master 127.29.243.254:44667 was elected leader, sending a full tablet report...
I20260809 04:55:57.760701 32158 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:45617
I20260809 04:55:57.760833 32359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:45617 every 8 connection(s)
I20260809 04:55:57.762725 32158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:55:57.765689 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 32158
I20260809 04:55:57.765978 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:55:57.766582 30671 external_mini_cluster.cc:1428] Setting key f9d5e00d8590b1d8d6b2dca09e70c1fb
Aug 09 04:55:57 dist-test-slave-21b9 krb5kdc[31553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251357, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:55:57.796128 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:57.766711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:40789 (local address 127.29.243.254:44667)
0809 04:55:57.767694 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:57.767701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:57.769545 (+ 1844us) server_negotiation.cc:407] Connection header received
0809 04:55:57.773057 (+ 3512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:57.773062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:57.773158 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:57.773251 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:57.775386 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.776610 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.778768 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.779498 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.779617 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:57.787382 (+ 7765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:57.787389 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:57.787393 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:57.787431 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:57.791954 (+ 4523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:57.792943 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:57.792948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:57.792950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:57.793133 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:57.793721 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:57.793725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:57.793727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:57.794124 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:57.794385 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:57.795187 (+ 802us) server_negotiation.cc:299] Negotiation successful
0809 04:55:57.795455 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":629,"threads_started":1}
I20260809 04:55:57.797379 32366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:57.767561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44667 (local address 127.29.243.195:40789)
0809 04:55:57.769379 (+ 1818us) negotiation.cc:107] Waiting for socket to connect
0809 04:55:57.769430 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 04:55:57.772790 (+ 3360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:55:57.773551 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:55:57.773560 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:55:57.774303 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:55:57.775198 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:57.775218 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.776793 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.776797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:57.778629 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:57.778643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.779904 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.779913 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:57.780239 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:57.783544 (+ 3305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:55:57.783602 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:55:57.787123 (+ 3521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:55:57.792101 (+ 4978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:57.792111 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:57.792128 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:57.792816 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:57.793284 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:55:57.793288 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:55:57.793290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:55:57.793600 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:55:57.794336 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:55:57.794345 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:55:57.794752 (+ 407us) client_negotiation.cc:769] Sending connection context
0809 04:55:57.795043 (+ 291us) client_negotiation.cc:240] Negotiation successful
0809 04:55:57.795777 (+ 734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1366,"thread_start_us":968,"threads_started":1}
I20260809 04:55:57.798889 32360 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44667
I20260809 04:55:57.799261 32360 heartbeater.cc:464] Registering TS with master...
I20260809 04:55:57.800398 32360 heartbeater.cc:511] Master 127.29.243.254:44667 requested a full tablet report, sending...
I20260809 04:55:57.802213 31628 ts_manager.cc:194] Registered new tserver with Master: 7ac2ed5251504c7385f1f946e29af4c2 (127.29.243.195:45617)
I20260809 04:55:57.803324 31628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:40789
I20260809 04:55:57.808063 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:55:57.836854 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:57.817564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.29.243.254:44667)
0809 04:55:57.817718 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:57.817722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:57.818941 (+ 1219us) server_negotiation.cc:407] Connection header received
0809 04:55:57.819152 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:57.819156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:57.819223 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:57.819345 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:57.822200 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.823319 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.825102 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.825766 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.825900 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:57.829450 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:57.829456 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:57.829459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:57.829494 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:57.833417 (+ 3923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:57.834253 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:57.834258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:57.834260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:57.834432 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:57.834894 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:57.834898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:57.834900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:57.835412 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:57.835699 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:57.836167 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 04:55:57.836431 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:55:57.846499 31628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:55:57.860098 31628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:55:57.904953 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:57.883866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51326 (local address 127.29.243.194:35849)
0809 04:55:57.884464 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:57.884473 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:57.886269 (+ 1796us) server_negotiation.cc:407] Connection header received
0809 04:55:57.886727 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:57.886736 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:57.887066 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:57.887311 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:55:57.889270 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.891362 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.896501 (+ 5139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.899097 (+ 2596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.899288 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:57.899644 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:57.904217 (+ 4573us) server_negotiation.cc:299] Negotiation successful
0809 04:55:57.904465 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":339,"threads_started":1}
I20260809 04:55:57.905869 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:57.884700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50840 (local address 127.29.243.193:41279)
0809 04:55:57.885459 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:57.885467 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:57.886570 (+ 1103us) server_negotiation.cc:407] Connection header received
0809 04:55:57.886793 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:57.886799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:57.887000 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:57.887193 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:55:57.888475 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.891608 (+ 3133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.897645 (+ 6037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.904078 (+ 6433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.904288 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:57.904476 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:57.905368 (+ 892us) server_negotiation.cc:299] Negotiation successful
0809 04:55:57.905541 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":390,"threads_started":1}
I20260809 04:55:57.906073 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:57.883358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:35849 (local address 127.0.0.1:51326)
0809 04:55:57.885998 (+ 2640us) negotiation.cc:107] Waiting for socket to connect
0809 04:55:57.886060 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 04:55:57.886449 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:55:57.887681 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:55:57.887686 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:55:57.887720 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:55:57.889022 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:57.889039 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.891612 (+ 2573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.891620 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:57.896252 (+ 4632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:57.896270 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.903581 (+ 7311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.903592 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:57.903929 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:57.903994 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 04:55:57.904127 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 04:55:57.904345 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2216,"mutex_wait_us":1,"thread_start_us":1034,"threads_started":1}
I20260809 04:55:57.906457 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:57.884705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:41279 (local address 127.0.0.1:50840)
0809 04:55:57.886460 (+ 1755us) negotiation.cc:107] Waiting for socket to connect
0809 04:55:57.886474 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:55:57.886623 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:55:57.887558 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:55:57.887565 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:55:57.887598 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:55:57.888236 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:57.888252 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.891787 (+ 3535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.891793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:57.897348 (+ 5555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:57.897366 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.904740 (+ 7374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.904747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:57.904942 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:57.904976 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 04:55:57.905070 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 04:55:57.905200 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1586,"thread_start_us":1895,"threads_started":1}
I20260809 04:55:57.913754 32045 tablet_service.cc:1467] Processing CreateTablet for tablet 1e258a9223e049a8aead38bf58ae06a6 (DEFAULT_TABLE table=test-table [id=02fa60fb68d74795a57df95b4c275953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:55:57.913863 31833 tablet_service.cc:1467] Processing CreateTablet for tablet 1e258a9223e049a8aead38bf58ae06a6 (DEFAULT_TABLE table=test-table [id=02fa60fb68d74795a57df95b4c275953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:55:57.915696 32045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e258a9223e049a8aead38bf58ae06a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:55:57.916078 31833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e258a9223e049a8aead38bf58ae06a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:55:57.917376 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:57.887070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:45617 (local address 127.0.0.1:37652)
0809 04:55:57.902483 (+ 15413us) negotiation.cc:107] Waiting for socket to connect
0809 04:55:57.902502 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:55:57.902668 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:55:57.903514 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:55:57.903519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:55:57.903546 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:55:57.904637 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:57.904652 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.908750 (+ 4098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.908755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:57.912697 (+ 3942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:57.912715 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.916071 (+ 3356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.916081 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:57.916456 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:57.916497 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 04:55:57.916583 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:55:57.916718 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15215,"thread_start_us":14861,"threads_started":1}
I20260809 04:55:57.918825 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:57.887486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37652 (local address 127.29.243.195:45617)
0809 04:55:57.901709 (+ 14223us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:57.901725 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:57.902801 (+ 1076us) server_negotiation.cc:407] Connection header received
0809 04:55:57.902952 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:57.902961 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:57.903233 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:57.904627 (+ 1394us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:55:57.905632 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.908562 (+ 2930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.912976 (+ 4414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:57.915419 (+ 2443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:57.915738 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:57.917953 (+ 2215us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:57.918096 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 04:55:57.918302 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13939,"thread_start_us":13493,"threads_started":1}
I20260809 04:55:57.924548 32252 tablet_service.cc:1467] Processing CreateTablet for tablet 1e258a9223e049a8aead38bf58ae06a6 (DEFAULT_TABLE table=test-table [id=02fa60fb68d74795a57df95b4c275953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:55:57.926781 32252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e258a9223e049a8aead38bf58ae06a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:55:57.937851 32391 tablet_bootstrap.cc:492] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e: Bootstrap starting.
I20260809 04:55:57.940843 32389 tablet_bootstrap.cc:492] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5: Bootstrap starting.
I20260809 04:55:57.943976 32389 tablet_bootstrap.cc:654] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5: Neither blocks nor log segments found. Creating new log.
I20260809 04:55:57.945492 32389 log.cc:824] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5: Log is configured to *not* fsync() on all Append() calls
I20260809 04:55:57.945799 32391 tablet_bootstrap.cc:654] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e: Neither blocks nor log segments found. Creating new log.
I20260809 04:55:57.949786 32394 tablet_bootstrap.cc:492] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2: Bootstrap starting.
I20260809 04:55:57.950042 32391 log.cc:824] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:55:57.955193 32391 tablet_bootstrap.cc:492] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e: No bootstrap required, opened a new log
I20260809 04:55:57.955729 32391 ts_tablet_manager.cc:1397] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.004s
I20260809 04:55:57.959156 32394 tablet_bootstrap.cc:654] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2: Neither blocks nor log segments found. Creating new log.
I20260809 04:55:57.961644 32394 log.cc:824] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2: Log is configured to *not* fsync() on all Append() calls
I20260809 04:55:57.966634 32389 tablet_bootstrap.cc:492] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5: No bootstrap required, opened a new log
I20260809 04:55:57.967079 32389 ts_tablet_manager.cc:1397] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.000s
I20260809 04:55:57.969642 32394 tablet_bootstrap.cc:492] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2: No bootstrap required, opened a new log
I20260809 04:55:57.970199 32394 ts_tablet_manager.cc:1397] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.007s
I20260809 04:55:57.972491 32391 raft_consensus.cc:348] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35849 } } peers { permanent_uuid: "eb124a8217924048973b2c065b72d0e5" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41279 } } peers { permanent_uuid: "7ac2ed5251504c7385f1f946e29af4c2" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45617 } }
I20260809 04:55:57.973224 32391 raft_consensus.cc:374] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:55:57.973428 32391 raft_consensus.cc:729] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf0cb4a6e8e445c87ce5a43230bab4e, State: Initialized, Role: FOLLOWER
I20260809 04:55:57.974319 32391 consensus_queue.cc:260] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35849 } } peers { permanent_uuid: "eb124a8217924048973b2c065b72d0e5" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41279 } } peers { permanent_uuid: "7ac2ed5251504c7385f1f946e29af4c2" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45617 } }
I20260809 04:55:57.976603 32146 heartbeater.cc:503] Master 127.29.243.254:44667 was elected leader, sending a full tablet report...
I20260809 04:55:57.977303 32391 ts_tablet_manager.cc:1428] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
I20260809 04:55:57.981506 32389 raft_consensus.cc:348] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35849 } } peers { permanent_uuid: "eb124a8217924048973b2c065b72d0e5" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41279 } } peers { permanent_uuid: "7ac2ed5251504c7385f1f946e29af4c2" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45617 } }
I20260809 04:55:57.982199 32389 raft_consensus.cc:374] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:55:57.982317 32389 raft_consensus.cc:729] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb124a8217924048973b2c065b72d0e5, State: Initialized, Role: FOLLOWER
I20260809 04:55:57.983196 32389 consensus_queue.cc:260] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35849 } } peers { permanent_uuid: "eb124a8217924048973b2c065b72d0e5" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41279 } } peers { permanent_uuid: "7ac2ed5251504c7385f1f946e29af4c2" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45617 } }
I20260809 04:55:57.986208 32389 ts_tablet_manager.cc:1428] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5: Time spent starting tablet: real 0.019s user 0.016s sys 0.004s
I20260809 04:55:57.987433 32394 raft_consensus.cc:348] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35849 } } peers { permanent_uuid: "eb124a8217924048973b2c065b72d0e5" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41279 } } peers { permanent_uuid: "7ac2ed5251504c7385f1f946e29af4c2" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45617 } }
I20260809 04:55:57.988157 32394 raft_consensus.cc:374] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:55:57.988346 32394 raft_consensus.cc:729] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ac2ed5251504c7385f1f946e29af4c2, State: Initialized, Role: FOLLOWER
I20260809 04:55:57.989225 32394 consensus_queue.cc:260] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35849 } } peers { permanent_uuid: "eb124a8217924048973b2c065b72d0e5" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41279 } } peers { permanent_uuid: "7ac2ed5251504c7385f1f946e29af4c2" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45617 } }
I20260809 04:55:57.991259 32360 heartbeater.cc:503] Master 127.29.243.254:44667 was elected leader, sending a full tablet report...
I20260809 04:55:57.992100 32394 ts_tablet_manager.cc:1428] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2: Time spent starting tablet: real 0.022s user 0.012s sys 0.007s
W20260809 04:55:58.015954 32363 tablet.cc:2369] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:55:58.088187 32400 raft_consensus.cc:482] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:55:58.088856 32400 raft_consensus.cc:504] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35849 } } peers { permanent_uuid: "eb124a8217924048973b2c065b72d0e5" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41279 } } peers { permanent_uuid: "7ac2ed5251504c7385f1f946e29af4c2" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45617 } }
I20260809 04:55:58.092280 32400 leader_election.cc:288] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf0cb4a6e8e445c87ce5a43230bab4e (127.29.243.194:35849), 7ac2ed5251504c7385f1f946e29af4c2 (127.29.243.195:45617)
I20260809 04:55:58.104521 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:58.093110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:52613 (local address 127.29.243.194:35849)
0809 04:55:58.093351 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:58.093355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:58.095161 (+ 1806us) server_negotiation.cc:407] Connection header received
0809 04:55:58.095385 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:58.095391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:58.095474 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:58.095573 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:55:58.096826 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.097901 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.101325 (+ 3424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.103086 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.103206 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:58.103263 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:58.103961 (+ 698us) server_negotiation.cc:299] Negotiation successful
0809 04:55:58.104102 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:55:58.104776 32406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:58.092401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:35849 (local address 127.29.243.193:52613)
0809 04:55:58.095047 (+ 2646us) negotiation.cc:107] Waiting for socket to connect
0809 04:55:58.095066 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:55:58.095222 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:55:58.095592 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:55:58.095597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:55:58.095626 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:55:58.096667 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:58.096679 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.098027 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.098032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:58.101172 (+ 3140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:58.101185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.103487 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.103496 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:58.103759 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:58.103795 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 04:55:58.103882 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 04:55:58.104027 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2450,"mutex_wait_us":1,"thread_start_us":1818,"threads_started":1}
I20260809 04:55:58.105232 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:58.092789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:53079 (local address 127.29.243.195:45617)
0809 04:55:58.093067 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:58.093074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:58.095404 (+ 2330us) server_negotiation.cc:407] Connection header received
0809 04:55:58.095652 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:58.095657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:58.095738 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:58.095838 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:55:58.097376 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.098887 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.102273 (+ 3386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.103790 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.103984 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:58.104058 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:58.104787 (+ 729us) server_negotiation.cc:299] Negotiation successful
0809 04:55:58.104921 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 04:55:58.106210 32408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:58.092618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:45617 (local address 127.29.243.193:53079)
0809 04:55:58.095304 (+ 2686us) negotiation.cc:107] Waiting for socket to connect
0809 04:55:58.095323 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:55:58.095513 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:55:58.095876 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:55:58.095879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:55:58.095917 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:55:58.097175 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:58.097191 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.099113 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.099122 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:58.102110 (+ 2988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:55:58.102126 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.104241 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.104250 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:55:58.104581 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:58.104616 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 04:55:58.105427 (+ 811us) client_negotiation.cc:240] Negotiation successful
0809 04:55:58.105527 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2478,"mutex_wait_us":16,"spinlock_wait_cycles":21248,"thread_start_us":1904,"threads_started":1}
I20260809 04:55:58.106422 32079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e258a9223e049a8aead38bf58ae06a6" candidate_uuid: "eb124a8217924048973b2c065b72d0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e" is_pre_election: true
I20260809 04:55:58.107723 32079 raft_consensus.cc:2454] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb124a8217924048973b2c065b72d0e5 in term 0.
I20260809 04:55:58.107577 32287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e258a9223e049a8aead38bf58ae06a6" candidate_uuid: "eb124a8217924048973b2c065b72d0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ac2ed5251504c7385f1f946e29af4c2" is_pre_election: true
I20260809 04:55:58.108601 32287 raft_consensus.cc:2454] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb124a8217924048973b2c065b72d0e5 in term 0.
I20260809 04:55:58.108983 31753 leader_election.cc:302] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccf0cb4a6e8e445c87ce5a43230bab4e, eb124a8217924048973b2c065b72d0e5; no voters:
I20260809 04:55:58.109580 32400 raft_consensus.cc:2781] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:55:58.109917 32400 raft_consensus.cc:482] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:55:58.110147 32400 raft_consensus.cc:3037] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:55:58.113888 32400 raft_consensus.cc:504] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35849 } } peers { permanent_uuid: "eb124a8217924048973b2c065b72d0e5" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41279 } } peers { permanent_uuid: "7ac2ed5251504c7385f1f946e29af4c2" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45617 } }
I20260809 04:55:58.114856 32400 leader_election.cc:288] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [CANDIDATE]: Term 1 election: Requested vote from peers ccf0cb4a6e8e445c87ce5a43230bab4e (127.29.243.194:35849), 7ac2ed5251504c7385f1f946e29af4c2 (127.29.243.195:45617)
I20260809 04:55:58.115294 32079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e258a9223e049a8aead38bf58ae06a6" candidate_uuid: "eb124a8217924048973b2c065b72d0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e"
I20260809 04:55:58.115471 32287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e258a9223e049a8aead38bf58ae06a6" candidate_uuid: "eb124a8217924048973b2c065b72d0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ac2ed5251504c7385f1f946e29af4c2"
I20260809 04:55:58.115566 32079 raft_consensus.cc:3037] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:55:58.115671 32287 raft_consensus.cc:3037] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:55:58.119138 32287 raft_consensus.cc:2454] T 1e258a9223e049a8aead38bf58ae06a6 P 7ac2ed5251504c7385f1f946e29af4c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb124a8217924048973b2c065b72d0e5 in term 1.
I20260809 04:55:58.119328 32079 raft_consensus.cc:2454] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb124a8217924048973b2c065b72d0e5 in term 1.
I20260809 04:55:58.119689 31760 leader_election.cc:302] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ac2ed5251504c7385f1f946e29af4c2, eb124a8217924048973b2c065b72d0e5; no voters:
I20260809 04:55:58.120106 32400 raft_consensus.cc:2781] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:55:58.121052 32400 raft_consensus.cc:686] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [term 1 LEADER]: Becoming Leader. State: Replica: eb124a8217924048973b2c065b72d0e5, State: Running, Role: LEADER
I20260809 04:55:58.121697 32400 consensus_queue.cc:237] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35849 } } peers { permanent_uuid: "eb124a8217924048973b2c065b72d0e5" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41279 } } peers { permanent_uuid: "7ac2ed5251504c7385f1f946e29af4c2" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45617 } }
I20260809 04:55:58.128620 31628 catalog_manager.cc:5495] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb124a8217924048973b2c065b72d0e5 (127.29.243.193). New cstate: current_term: 1 leader_uuid: "eb124a8217924048973b2c065b72d0e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf0cb4a6e8e445c87ce5a43230bab4e" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb124a8217924048973b2c065b72d0e5" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ac2ed5251504c7385f1f946e29af4c2" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45617 } health_report { overall_health: UNKNOWN } } }
W20260809 04:55:58.150290 31936 tablet.cc:2369] T 1e258a9223e049a8aead38bf58ae06a6 P eb124a8217924048973b2c065b72d0e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 04:55:58 dist-test-slave-21b9 krb5kdc[31553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:55:58.161405 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:58.138411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.29.243.193:41279)
0809 04:55:58.138670 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:58.138675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:58.139836 (+ 1161us) server_negotiation.cc:407] Connection header received
0809 04:55:58.140064 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:58.140069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:58.140184 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:58.140275 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:58.143579 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.144640 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.146510 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.147190 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.147330 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:58.153734 (+ 6404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:58.153747 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:58.153763 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:58.153802 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:58.157783 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.158548 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.158556 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.158558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.158870 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.159368 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.159372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.159375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.159895 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:58.160284 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:58.160599 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 04:55:58.160915 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 04:55:58.167382 32147 tablet.cc:2369] T 1e258a9223e049a8aead38bf58ae06a6 P ccf0cb4a6e8e445c87ce5a43230bab4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 04:55:58 dist-test-slave-21b9 krb5kdc[31553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:55:58 dist-test-slave-21b9 krb5kdc[31553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:55:58.223630 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:58.205094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50858 (local address 127.29.243.193:41279)
0809 04:55:58.205325 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:58.205330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:58.205492 (+ 162us) server_negotiation.cc:407] Connection header received
0809 04:55:58.205723 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:58.205728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:58.205801 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:58.205893 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:58.207920 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.208678 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.210597 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.211087 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.211196 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:58.217177 (+ 5981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:58.217183 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:58.217187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:58.217227 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:58.220341 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.221193 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.221197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.221200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.221421 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.221873 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.221877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.221879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.222276 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:58.222715 (+ 439us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:58.222836 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 04:55:58.223067 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
Aug 09 04:55:58 dist-test-slave-21b9 krb5kdc[31553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:55:58 dist-test-slave-21b9 krb5kdc[31553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:55:58.282101 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:58.261366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.29.243.193:41279)
0809 04:55:58.261646 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:58.261653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:58.262189 (+ 536us) server_negotiation.cc:407] Connection header received
0809 04:55:58.262391 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:58.262396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:58.262467 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:58.262568 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:58.265203 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.266109 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.268133 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.268955 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.269093 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:58.275153 (+ 6060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:58.275159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:58.275163 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:58.275204 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:58.279239 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.279908 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.279913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.279915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.280131 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.280474 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.280479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.280481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.280911 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:58.281269 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:58.281389 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 04:55:58.281562 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
W20260809 04:55:58.284883 31802 tablet_service.cc:2908] Rejecting scan request for tablet 1e258a9223e049a8aead38bf58ae06a6: Uninitialized: clean time has not yet been initialized
I20260809 04:55:58.289178 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31729
I20260809 04:55:58.307631 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:58.290076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50890 (local address 127.29.243.193:41279)
0809 04:55:58.290241 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:58.290245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:58.292518 (+ 2273us) server_negotiation.cc:407] Connection header received
0809 04:55:58.292657 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:58.292661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:58.292722 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:58.292800 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:58.294516 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.295458 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.297113 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.297935 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.298043 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:58.301462 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:58.301467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:58.301470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:58.301507 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:58.304333 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.305155 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.305161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.305164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.305327 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.305709 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.305714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.305716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.305980 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:58.306389 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:58.306532 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 04:55:58.306996 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:55:58.308508 31928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:55:58.401098 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 31729
I20260809 04:55:58.427448 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31944
Aug 09 04:55:58 dist-test-slave-21b9 krb5kdc[31553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:55:58.448040 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:58.428385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51340 (local address 127.29.243.194:35849)
0809 04:55:58.428622 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:58.428628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:58.428646 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:55:58.428698 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:58.428701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:58.428762 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:58.428859 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:58.430729 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.431653 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.433530 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.433931 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.434052 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:58.442327 (+ 8275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:58.442343 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:58.442363 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:58.442401 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:58.445223 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.445897 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.445906 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.445908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.446045 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.446571 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.446575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.446577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.446991 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:58.447243 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:58.447592 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 04:55:58.447767 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 04:55:58.448845 32140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:55:58.538618 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 31944
I20260809 04:55:58.563077 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32158
Aug 09 04:55:58 dist-test-slave-21b9 krb5kdc[31553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:55:58.578776 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:55:58.563950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37656 (local address 127.29.243.195:45617)
0809 04:55:58.564157 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:55:58.564163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:55:58.564186 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:55:58.564287 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:55:58.564291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:55:58.564360 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:55:58.564455 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:55:58.565716 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.566425 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.567723 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:55:58.568104 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:55:58.568218 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:55:58.572152 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:55:58.572168 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:55:58.572186 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:55:58.572218 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:55:58.575746 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.576502 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.576512 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.576514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.576659 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:55:58.577077 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:55:58.577082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:55:58.577084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:55:58.577504 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:55:58.577815 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 04:55:58.578156 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 04:55:58.578397 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:55:58.579566 32350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:55:58.665711 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 32158
I20260809 04:55:58.690701 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31578
I20260809 04:55:58.691926 31678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:55:58.800420 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 31578
2026-08-09T04:55:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4611 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:55:58 dist-test-slave-21b9 krb5kdc[32436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:55:58 dist-test-slave-21b9 krb5kdc[32436](info): set up 2 sockets
Aug 09 04:55:58 dist-test-slave-21b9 krb5kdc[32436](info): commencing operation
krb5kdc: starting...
W20260809 04:56:00.002629 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:56:00 dist-test-slave-21b9 krb5kdc[32436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:56:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:56:00.206243 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:37499
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:37499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:00.553187 32462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:00.553476 32462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:00.553601 32462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:00.561049 32462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:00.561168 32462 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:56:00.561224 32462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:00.561299 32462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:00.561358 32462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:00.561430 32462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:00.571627 32462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:37499
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:37499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:00.572927 32462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:00.574292 32462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:00.586297 32478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:00.586357 32472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:00.586405 32473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:00.592733 32462 server_base.cc:1034] running on GCE node
I20260809 04:56:00.593482 32462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:00.594908 32462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:00.596127 32462 hybrid_clock.cc:648] HybridClock initialized: now 1786251360596061 us; error 33 us; skew 500 ppm
I20260809 04:56:00.596460 32462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:00 dist-test-slave-21b9 krb5kdc[32436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251360, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:00.599486 32462 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:56:00.601038 32462 webserver.cc:460] Webserver started at http://127.29.243.254:46355/ using document root <none> and password file <none>
I20260809 04:56:00.601749 32462 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:00.601840 32462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:00.602169 32462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:00.605142 32462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "dd82eae58d2145698a80ac43b6f4e15d"
format_stamp: "Formatted at 2026-08-09 04:56:00 on dist-test-slave-21b9"
server_key: "009f29ff9c67ce6b98ea070e646ef67d"
server_key_iv: "e23972dbc94a5b77d976e287dec04012"
server_key_version: "encryptionkey@0"
I20260809 04:56:00.605886 32462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "dd82eae58d2145698a80ac43b6f4e15d"
format_stamp: "Formatted at 2026-08-09 04:56:00 on dist-test-slave-21b9"
server_key: "009f29ff9c67ce6b98ea070e646ef67d"
server_key_iv: "e23972dbc94a5b77d976e287dec04012"
server_key_version: "encryptionkey@0"
I20260809 04:56:00.611263 32462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 04:56:00.614703 32487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:00.616040 32462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:00.616195 32462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "dd82eae58d2145698a80ac43b6f4e15d"
format_stamp: "Formatted at 2026-08-09 04:56:00 on dist-test-slave-21b9"
server_key: "009f29ff9c67ce6b98ea070e646ef67d"
server_key_iv: "e23972dbc94a5b77d976e287dec04012"
server_key_version: "encryptionkey@0"
I20260809 04:56:00.616397 32462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:00.631551 32462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:00.635430 32462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:00.635658 32462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:00.657714 32462 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:37499
I20260809 04:56:00.657733 32583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:37499 every 8 connection(s)
I20260809 04:56:00.659086 32462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:00.662904 32584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:00.667796 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 32462
I20260809 04:56:00.668063 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:00.668680 30671 external_mini_cluster.cc:1428] Setting key 2ab503d5b64de441b2c02d244e44dc57
I20260809 04:56:00.673906 32584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d: Bootstrap starting.
I20260809 04:56:00.677721 32584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:00.679001 32584 log.cc:824] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:00.682602 32584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d: No bootstrap required, opened a new log
Aug 09 04:56:00 dist-test-slave-21b9 krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:00.691566 32584 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd82eae58d2145698a80ac43b6f4e15d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37499 } }
I20260809 04:56:00.692065 32584 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:00.692191 32584 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd82eae58d2145698a80ac43b6f4e15d, State: Initialized, Role: FOLLOWER
I20260809 04:56:00.692909 32584 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd82eae58d2145698a80ac43b6f4e15d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37499 } }
I20260809 04:56:00.693135 32584 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:00.693307 32584 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:00.693480 32584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:00.693938 32588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:00.670754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53124 (local address 127.29.243.254:37499)
0809 04:56:00.671686 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:00.671698 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:00.671744 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:56:00.673494 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:00.673518 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:00.673904 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:00.674262 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:00.676065 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:00.677372 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:00.679115 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:00.680225 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:00.680460 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:00.685394 (+ 4934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:00.685403 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:00.685423 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:00.685462 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:00.689333 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:00.690124 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:00.690130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:00.690137 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:00.690336 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:00.690810 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:00.690822 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:00.690824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:00.691447 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:00.691695 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:00.692261 (+ 566us) server_negotiation.cc:299] Negotiation successful
0809 04:56:00.692716 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":422,"threads_started":1}
I20260809 04:56:00.695923 32584 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd82eae58d2145698a80ac43b6f4e15d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37499 } }
I20260809 04:56:00.696416 32584 leader_election.cc:302] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd82eae58d2145698a80ac43b6f4e15d; no voters:
I20260809 04:56:00.697039 32584 leader_election.cc:288] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:00.697147 32591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:00.697803 32591 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [term 1 LEADER]: Becoming Leader. State: Replica: dd82eae58d2145698a80ac43b6f4e15d, State: Running, Role: LEADER
I20260809 04:56:00.698369 32591 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd82eae58d2145698a80ac43b6f4e15d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37499 } }
I20260809 04:56:00.699254 32584 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:00.701558 32592 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd82eae58d2145698a80ac43b6f4e15d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd82eae58d2145698a80ac43b6f4e15d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37499 } } }
I20260809 04:56:00.701522 32596 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd82eae58d2145698a80ac43b6f4e15d. Latest consensus state: current_term: 1 leader_uuid: "dd82eae58d2145698a80ac43b6f4e15d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd82eae58d2145698a80ac43b6f4e15d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37499 } } }
I20260809 04:56:00.701973 32596 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:00.701972 32592 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:00.703711 32606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:00.708721 32606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:00.717010 32606 catalog_manager.cc:1347] Generated new cluster ID: ec1efda97ee64f8eb5bc6019415ed9fa
I20260809 04:56:00.717118 32606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:00.748772 32606 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:00.750083 32606 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:00.759810 32606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d: Generated new TSK 0
I20260809 04:56:00.760838 32606 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:56:00.818379 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:37499
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:01.154300 32630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:01.154599 32630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:01.154747 32630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:01.161785 32630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:01.161912 32630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:01.162045 32630 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:56:01.162159 32630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:56:01.171730 32630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.243.254:37499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:01.173040 32630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:01.174371 32630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:01.187405 32644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:01.187502 32636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:01.187502 32638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:01.188233 32630 server_base.cc:1034] running on GCE node
I20260809 04:56:01.188804 32630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:01.189666 32630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:01.190887 32630 hybrid_clock.cc:648] HybridClock initialized: now 1786251361190824 us; error 49 us; skew 500 ppm
I20260809 04:56:01.191179 32630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:01 dist-test-slave-21b9 krb5kdc[32436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251361, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:01.194221 32630 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:56:01.195765 32630 webserver.cc:460] Webserver started at http://127.29.243.193:43437/ using document root <none> and password file <none>
I20260809 04:56:01.196451 32630 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:01.196563 32630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:01.196880 32630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:01.199858 32630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "c34f53fb768143e88b356cd137c36b8f"
format_stamp: "Formatted at 2026-08-09 04:56:01 on dist-test-slave-21b9"
server_key: "6d3c75c32da7d32a56682b675cb2f5cd"
server_key_iv: "fb3bac8df379c21fa654413a84d4d2ff"
server_key_version: "encryptionkey@0"
I20260809 04:56:01.200647 32630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "c34f53fb768143e88b356cd137c36b8f"
format_stamp: "Formatted at 2026-08-09 04:56:01 on dist-test-slave-21b9"
server_key: "6d3c75c32da7d32a56682b675cb2f5cd"
server_key_iv: "fb3bac8df379c21fa654413a84d4d2ff"
server_key_version: "encryptionkey@0"
I20260809 04:56:01.205956 32630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:56:01.209378 32654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:01.210872 32630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:01.211041 32630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "c34f53fb768143e88b356cd137c36b8f"
format_stamp: "Formatted at 2026-08-09 04:56:01 on dist-test-slave-21b9"
server_key: "6d3c75c32da7d32a56682b675cb2f5cd"
server_key_iv: "fb3bac8df379c21fa654413a84d4d2ff"
server_key_version: "encryptionkey@0"
I20260809 04:56:01.211248 32630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:01.236810 32630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:01.241004 32630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:01.241268 32630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:01.244197 32630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:01.244303 32630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:01.244438 32630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:01.244532 32630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:56:01 dist-test-slave-21b9 krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251361, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:01.267470 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:01.244757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:55389 (local address 127.29.243.254:37499)
0809 04:56:01.245519 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:01.245524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:01.246524 (+ 1000us) server_negotiation.cc:407] Connection header received
0809 04:56:01.248578 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:01.248584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:01.248667 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:01.248781 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:01.251205 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.252162 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.255027 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.255623 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.255761 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:01.260719 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:01.260727 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:01.260732 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:01.260777 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:01.263898 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:01.264835 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:01.264840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:01.264844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:01.265020 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:01.265598 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:01.265603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:01.265607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:01.265973 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:01.266107 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:01.266838 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 04:56:01.267083 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":551,"threads_started":1}
I20260809 04:56:01.268708 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:01.245483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37499 (local address 127.29.243.193:55389)
0809 04:56:01.246338 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:01.246384 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:01.248287 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:01.249154 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:01.249169 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:01.250020 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:01.250970 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:01.251014 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.252358 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.252364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:01.254856 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:01.254872 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.255887 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.255899 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:01.256215 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:01.257178 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:01.257241 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:01.260443 (+ 3202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:01.264053 (+ 3610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:01.264062 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:01.264082 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:01.264696 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:01.265168 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:01.265173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:01.265177 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:01.265437 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:01.266116 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:01.266125 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:01.266478 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 04:56:01.266724 (+ 246us) client_negotiation.cc:240] Negotiation successful
0809 04:56:01.267232 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":377,"threads_started":1}
I20260809 04:56:01.297974 32630 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:44961
I20260809 04:56:01.298017 374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:44961 every 8 connection(s)
I20260809 04:56:01.299650 32630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:56:01.302331 375 heartbeater.cc:347] Connected to a master server at 127.29.243.254:37499
I20260809 04:56:01.302580 375 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:01.303452 375 heartbeater.cc:511] Master 127.29.243.254:37499 requested a full tablet report, sending...
I20260809 04:56:01.305078 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:01.290007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:36101 (local address 127.29.243.254:37499)
0809 04:56:01.290213 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:01.290220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:01.291151 (+ 931us) server_negotiation.cc:407] Connection header received
0809 04:56:01.291398 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:01.291405 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:01.291500 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:01.291572 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:01.293275 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.294043 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.296144 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.296559 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.296659 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:01.299113 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:01.299119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:01.299122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:01.299155 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:01.302046 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:01.302817 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:01.302822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:01.302826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:01.302990 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:01.303442 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:01.303448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:01.303451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:01.303828 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:01.303986 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:01.304483 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 04:56:01.304711 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:56:01.305105 350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:01.289981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37499 (local address 127.29.243.193:36101)
0809 04:56:01.291028 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:01.291051 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:01.291217 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:01.291823 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:01.291828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:01.292564 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:01.293100 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:01.293117 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.294192 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.294198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:01.295945 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:01.295963 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.296775 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.296783 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:01.296985 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:01.297753 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:01.297782 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:01.298908 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:01.302222 (+ 3314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:01.302227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:01.302230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:01.302699 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:01.303137 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:01.303141 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:01.303144 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:01.303306 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:01.303970 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:01.303976 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:01.304175 (+ 199us) client_negotiation.cc:769] Sending connection context
0809 04:56:01.304308 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 04:56:01.304619 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"thread_start_us":655,"threads_started":1}
I20260809 04:56:01.305894 32509 ts_manager.cc:194] Registered new tserver with Master: c34f53fb768143e88b356cd137c36b8f (127.29.243.193:44961)
I20260809 04:56:01.307444 32509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:55389
I20260809 04:56:01.309389 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 32630
I20260809 04:56:01.309607 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:56:01.310298 30671 external_mini_cluster.cc:1428] Setting key 47165fe9078df9007c42014d7698dfe7
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:56:01.353958 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:37499
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:01.680961 381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:01.681246 381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:01.681393 381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:01.688661 381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:01.688781 381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:01.688903 381 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:56:01.689020 381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:56:01.698783 381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.243.254:37499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:01.700203 381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:01.701617 381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:01.714576 393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:01.714531 389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:01.714617 390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:01.715528 381 server_base.cc:1034] running on GCE node
I20260809 04:56:01.716146 381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:01.717002 381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:01.718215 381 hybrid_clock.cc:648] HybridClock initialized: now 1786251361718144 us; error 45 us; skew 500 ppm
I20260809 04:56:01.718506 381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:01 dist-test-slave-21b9 krb5kdc[32436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251361, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:01.721714 381 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:56:01.723120 381 webserver.cc:460] Webserver started at http://127.29.243.194:45579/ using document root <none> and password file <none>
I20260809 04:56:01.723841 381 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:01.723944 381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:01.724212 381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:01.727089 381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "afd73c2e039542929dd0c4d82005c256"
format_stamp: "Formatted at 2026-08-09 04:56:01 on dist-test-slave-21b9"
server_key: "1b01a6beb85b69bb423d0b98c0dc65c3"
server_key_iv: "240aa972cde97a68e387b562477c543e"
server_key_version: "encryptionkey@0"
I20260809 04:56:01.727906 381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "afd73c2e039542929dd0c4d82005c256"
format_stamp: "Formatted at 2026-08-09 04:56:01 on dist-test-slave-21b9"
server_key: "1b01a6beb85b69bb423d0b98c0dc65c3"
server_key_iv: "240aa972cde97a68e387b562477c543e"
server_key_version: "encryptionkey@0"
I20260809 04:56:01.733086 381 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:56:01.736454 405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:01.737850 381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:56:01.738018 381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "afd73c2e039542929dd0c4d82005c256"
format_stamp: "Formatted at 2026-08-09 04:56:01 on dist-test-slave-21b9"
server_key: "1b01a6beb85b69bb423d0b98c0dc65c3"
server_key_iv: "240aa972cde97a68e387b562477c543e"
server_key_version: "encryptionkey@0"
I20260809 04:56:01.738222 381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:01.779978 381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:01.784396 381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:01.784655 381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:01.787230 381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:01.787339 381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:01.787474 381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:01.787534 381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:56:01 dist-test-slave-21b9 krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251361, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:01.810534 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:01.788012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:60877 (local address 127.29.243.254:37499)
0809 04:56:01.788210 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:01.788217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:01.789659 (+ 1442us) server_negotiation.cc:407] Connection header received
0809 04:56:01.791953 (+ 2294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:01.791960 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:01.792059 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:01.792177 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:01.794264 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.795196 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.797648 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.798256 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.798383 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:01.803364 (+ 4981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:01.803371 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:01.803376 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:01.803433 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:01.806700 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:01.807701 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:01.807707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:01.807711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:01.807869 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:01.808432 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:01.808437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:01.808440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:01.808822 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:01.808975 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:01.809948 (+ 973us) server_negotiation.cc:299] Negotiation successful
0809 04:56:01.810185 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 04:56:01.812026 419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:01.788590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37499 (local address 127.29.243.194:60877)
0809 04:56:01.789456 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:01.789524 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:01.791640 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:01.792443 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:01.792450 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:01.793168 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:01.794081 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:01.794109 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.795400 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.795405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:01.797431 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:01.797447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.798526 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.798538 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:01.798837 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:01.799625 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:01.799659 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:01.803011 (+ 3352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:01.806849 (+ 3838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:01.806863 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:01.806882 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:01.807568 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:01.807979 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:01.807985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:01.807989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:01.808262 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:01.809052 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:01.809058 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:01.809463 (+ 405us) client_negotiation.cc:769] Sending connection context
0809 04:56:01.809845 (+ 382us) client_negotiation.cc:240] Negotiation successful
0809 04:56:01.810450 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":376,"threads_started":1}
I20260809 04:56:01.831379 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:01.818136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:46689 (local address 127.29.243.254:37499)
0809 04:56:01.818363 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:01.818369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:01.818848 (+ 479us) server_negotiation.cc:407] Connection header received
0809 04:56:01.818989 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:01.818994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:01.819073 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:01.819178 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:01.820086 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.821282 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.822653 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.823206 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.823353 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:01.825561 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:01.825566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:01.825569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:01.825602 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:01.828739 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:01.829401 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:01.829407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:01.829411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:01.829549 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:01.829894 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:01.829899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:01.829902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:01.830248 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:01.830386 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:01.830798 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 04:56:01.831016 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:56:01.831528 552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:01.818124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37499 (local address 127.29.243.194:46689)
0809 04:56:01.818759 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:01.818778 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:01.818891 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:01.819181 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:01.819183 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:01.819666 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:01.819958 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:01.819969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.821435 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.821439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:01.822505 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:01.822519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:01.823431 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:01.823439 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:01.823608 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:01.824263 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:01.824280 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:01.825371 (+ 1091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:01.828879 (+ 3508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:01.828885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:01.828890 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:01.829288 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:01.829659 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:01.829664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:01.829667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:01.829792 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:01.830364 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:01.830369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:01.830554 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 04:56:01.830675 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 04:56:01.831003 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":330,"threads_started":1}
I20260809 04:56:01.834539 381 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:39139
I20260809 04:56:01.834558 600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:39139 every 8 connection(s)
I20260809 04:56:01.836086 381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:56:01.838085 603 heartbeater.cc:347] Connected to a master server at 127.29.243.254:37499
I20260809 04:56:01.838307 603 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:01.838973 603 heartbeater.cc:511] Master 127.29.243.254:37499 requested a full tablet report, sending...
I20260809 04:56:01.840464 32509 ts_manager.cc:194] Registered new tserver with Master: afd73c2e039542929dd0c4d82005c256 (127.29.243.194:39139)
I20260809 04:56:01.841320 32509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:60877
I20260809 04:56:01.842470 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 381
I20260809 04:56:01.842650 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:56:01.843092 30671 external_mini_cluster.cc:1428] Setting key 312b8c9492714391681721b2eaf64fe9
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:56:01.887338 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:37499
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:02.219491 608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:02.219779 608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:02.219885 608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:02.226840 608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:02.226944 608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:02.227028 608 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:56:02.227111 608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:56:02.236773 608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.243.254:37499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:02.237995 608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:02.239353 608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:02.252209 618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:02.252295 622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:02.252240 619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:02.253103 608 server_base.cc:1034] running on GCE node
I20260809 04:56:02.253700 608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:02.254555 608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:02.255761 608 hybrid_clock.cc:648] HybridClock initialized: now 1786251362255698 us; error 36 us; skew 500 ppm
I20260809 04:56:02.256038 608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:02 dist-test-slave-21b9 krb5kdc[32436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251362, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:02.259016 608 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:56:02.260535 608 webserver.cc:460] Webserver started at http://127.29.243.195:44967/ using document root <none> and password file <none>
I20260809 04:56:02.261235 608 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:02.261338 608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:02.261665 608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:02.264578 608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "39f7d35d953441fe9566e1f3ca088d89"
format_stamp: "Formatted at 2026-08-09 04:56:02 on dist-test-slave-21b9"
server_key: "ece0927d71250f212a7d0320be2fc07d"
server_key_iv: "7063160825a5ef94420b14874042851e"
server_key_version: "encryptionkey@0"
I20260809 04:56:02.265336 608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "39f7d35d953441fe9566e1f3ca088d89"
format_stamp: "Formatted at 2026-08-09 04:56:02 on dist-test-slave-21b9"
server_key: "ece0927d71250f212a7d0320be2fc07d"
server_key_iv: "7063160825a5ef94420b14874042851e"
server_key_version: "encryptionkey@0"
I20260809 04:56:02.270901 608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 04:56:02.274422 631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:02.275835 608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:02.275993 608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "39f7d35d953441fe9566e1f3ca088d89"
format_stamp: "Formatted at 2026-08-09 04:56:02 on dist-test-slave-21b9"
server_key: "ece0927d71250f212a7d0320be2fc07d"
server_key_iv: "7063160825a5ef94420b14874042851e"
server_key_version: "encryptionkey@0"
I20260809 04:56:02.276186 608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:02.306793 608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:02.310211 375 heartbeater.cc:503] Master 127.29.243.254:37499 was elected leader, sending a full tablet report...
I20260809 04:56:02.311050 608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:02.311398 608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:02.314603 608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:02.314749 608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 04:56:02.314888 608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:02.314965 608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:56:02 dist-test-slave-21b9 krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251362, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:02.341741 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.315363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:38821 (local address 127.29.243.254:37499)
0809 04:56:02.315534 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:02.315538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:02.317300 (+ 1762us) server_negotiation.cc:407] Connection header received
0809 04:56:02.319346 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:02.319352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:02.319441 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:02.319553 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:02.321777 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.322938 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.325871 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.326319 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.326431 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.331811 (+ 5380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:02.331819 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:02.331823 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:02.331912 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:02.336723 (+ 4811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:02.337727 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:02.337733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:02.337738 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:02.337986 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:02.338852 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:02.338858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:02.338862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:02.339446 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:02.339613 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:02.340922 (+ 1309us) server_negotiation.cc:299] Negotiation successful
0809 04:56:02.341224 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:56:02.343991 647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.316200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37499 (local address 127.29.243.195:38821)
0809 04:56:02.317128 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:02.317177 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:02.319046 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:02.319882 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:02.319895 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:02.320674 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:02.321557 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.321586 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.323116 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.323122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.325698 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.325720 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.326562 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.326570 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.326866 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.327753 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:02.327787 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:02.331397 (+ 3610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:02.336857 (+ 5460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:02.336867 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:02.336888 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:02.337558 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:02.338141 (+ 583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:02.338147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:02.338152 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:02.338643 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:02.339696 (+ 1053us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:02.339705 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:02.340332 (+ 627us) client_negotiation.cc:769] Sending connection context
0809 04:56:02.340684 (+ 352us) client_negotiation.cc:240] Negotiation successful
0809 04:56:02.341466 (+ 782us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":397,"threads_started":1}
I20260809 04:56:02.387089 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.361358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:60537 (local address 127.29.243.254:37499)
0809 04:56:02.361586 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:02.361592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:02.362993 (+ 1401us) server_negotiation.cc:407] Connection header received
0809 04:56:02.363295 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:02.363301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:02.363393 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:02.363510 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:02.366926 (+ 3416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.368252 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.370988 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.372047 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.372232 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.376857 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:02.376865 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:02.376871 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:02.376919 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:02.382138 (+ 5219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:02.383503 (+ 1365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:02.383509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:02.383512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:02.383823 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:02.384730 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:02.384735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:02.384738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:02.385214 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:02.385436 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:02.386251 (+ 815us) server_negotiation.cc:299] Negotiation successful
0809 04:56:02.386518 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 04:56:02.387473 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.361311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37499 (local address 127.29.243.195:60537)
0809 04:56:02.362819 (+ 1508us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:02.362841 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:02.363097 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:02.363576 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:02.363582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:02.365125 (+ 1543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:02.366716 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.366737 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.368508 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.368514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.370836 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.370857 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.372558 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.372568 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.373052 (+ 484us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.374237 (+ 1185us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:02.374264 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:02.376590 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:02.382315 (+ 5725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:02.382322 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:02.382328 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:02.383357 (+ 1029us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:02.383979 (+ 622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:02.383984 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:02.383988 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:02.384243 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:02.385522 (+ 1279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:02.385536 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:02.385925 (+ 389us) client_negotiation.cc:769] Sending connection context
0809 04:56:02.386099 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 04:56:02.386744 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1265,"thread_start_us":1028,"threads_started":1}
I20260809 04:56:02.419888 608 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:35159
I20260809 04:56:02.420015 831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:35159 every 8 connection(s)
I20260809 04:56:02.421530 608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:56:02.424997 833 heartbeater.cc:347] Connected to a master server at 127.29.243.254:37499
I20260809 04:56:02.425311 833 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:02.426596 833 heartbeater.cc:511] Master 127.29.243.254:37499 requested a full tablet report, sending...
I20260809 04:56:02.428380 32509 ts_manager.cc:194] Registered new tserver with Master: 39f7d35d953441fe9566e1f3ca088d89 (127.29.243.195:35159)
I20260809 04:56:02.429442 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 608
I20260809 04:56:02.429800 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:56:02.429811 32509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:38821
I20260809 04:56:02.430406 30671 external_mini_cluster.cc:1428] Setting key c6cab8575b0f250b0057290a9405ea57
I20260809 04:56:02.433770 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:56:02 dist-test-slave-21b9 krb5kdc[32436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251362, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:56:02 dist-test-slave-21b9 krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251362, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:02.494427 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.474578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53136 (local address 127.29.243.254:37499)
0809 04:56:02.474803 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:02.474809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:02.475119 (+ 310us) server_negotiation.cc:407] Connection header received
0809 04:56:02.475331 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:02.475336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:02.475435 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:02.475540 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:02.477621 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.478509 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.480520 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.481286 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.481424 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.487089 (+ 5665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:02.487094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:02.487097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:02.487134 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:02.490822 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:02.491603 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:02.491607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:02.491610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:02.491833 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:02.492369 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:02.492373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:02.492375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:02.492790 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:02.493085 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:02.493551 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 04:56:02.493796 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:56:02.499590 32509 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:02.511461 32509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:02.560925 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.529919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59156 (local address 127.29.243.193:44961)
0809 04:56:02.542715 (+ 12796us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:02.542725 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:02.544756 (+ 2031us) server_negotiation.cc:407] Connection header received
0809 04:56:02.545289 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:02.545297 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:02.545552 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:02.545820 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:02.548697 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.551128 (+ 2431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.556490 (+ 5362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.558914 (+ 2424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.559164 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.559447 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:02.560205 (+ 758us) server_negotiation.cc:299] Negotiation successful
0809 04:56:02.560429 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12598,"thread_start_us":12377,"threads_started":1}
I20260809 04:56:02.561043 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.530654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.29.243.195:35159)
0809 04:56:02.542110 (+ 11456us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:02.542122 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:02.544893 (+ 2771us) server_negotiation.cc:407] Connection header received
0809 04:56:02.545069 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:02.545079 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:02.545383 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:02.545674 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:02.548679 (+ 3005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.551941 (+ 3262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.556929 (+ 4988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.559130 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.559438 (+ 308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.559627 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:02.560247 (+ 620us) server_negotiation.cc:299] Negotiation successful
0809 04:56:02.560433 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11165,"thread_start_us":10685,"threads_started":1}
I20260809 04:56:02.561089 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.530162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:35159 (local address 127.0.0.1:50946)
0809 04:56:02.544373 (+ 14211us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:02.544391 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:02.544661 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:02.546220 (+ 1559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:02.546229 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:02.546321 (+ 92us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:02.548149 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.548168 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.552226 (+ 4058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.552232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.556666 (+ 4434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.556684 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.559530 (+ 2846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.559540 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.559904 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.559976 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 04:56:02.560153 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 04:56:02.560358 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13994,"mutex_wait_us":59,"spinlock_wait_cycles":2304,"thread_start_us":12973,"threads_started":1}
I20260809 04:56:02.562216 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.529564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:44961 (local address 127.0.0.1:59156)
0809 04:56:02.544041 (+ 14477us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:02.544095 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:02.544877 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:02.546218 (+ 1341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:02.546227 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:02.546277 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:02.548476 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.548490 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.551352 (+ 2862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.551359 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.556279 (+ 4920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.556293 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.559745 (+ 3452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.559757 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.560022 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.560049 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:56:02.561424 (+ 1375us) client_negotiation.cc:240] Negotiation successful
0809 04:56:02.561608 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13907,"mutex_wait_us":1,"spinlock_wait_cycles":4864,"thread_start_us":14185,"threads_started":1}
I20260809 04:56:02.566447 715 tablet_service.cc:1467] Processing CreateTablet for tablet 20896f58de314dcdbd40d3ad506d0407 (DEFAULT_TABLE table=test-table [id=c73df3c3c00e496587dd3cd628728ac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:02.568418 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.544418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:39139 (local address 127.0.0.1:46300)
0809 04:56:02.557287 (+ 12869us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:02.557307 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:02.557452 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:02.558607 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:02.558612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:02.558641 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:02.560020 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.560033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.562610 (+ 2577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.562614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.565499 (+ 2885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.565512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.567408 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.567418 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.567653 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.567678 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:56:02.567752 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 04:56:02.567880 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12711,"thread_start_us":12293,"threads_started":1}
I20260809 04:56:02.568470 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.544692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46300 (local address 127.29.243.194:39139)
0809 04:56:02.554558 (+ 9866us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:02.554565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:02.557434 (+ 2869us) server_negotiation.cc:407] Connection header received
0809 04:56:02.557633 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:02.557643 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:02.558004 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:02.558256 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:02.560328 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.562276 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.565678 (+ 3402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.567041 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.567258 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.567483 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:02.567841 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 04:56:02.568086 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9691,"thread_start_us":9481,"threads_started":1}
I20260809 04:56:02.568647 715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20896f58de314dcdbd40d3ad506d0407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:02.568001 32736 tablet_service.cc:1467] Processing CreateTablet for tablet 20896f58de314dcdbd40d3ad506d0407 (DEFAULT_TABLE table=test-table [id=c73df3c3c00e496587dd3cd628728ac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:02.569931 32736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20896f58de314dcdbd40d3ad506d0407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:02.572882 485 tablet_service.cc:1467] Processing CreateTablet for tablet 20896f58de314dcdbd40d3ad506d0407 (DEFAULT_TABLE table=test-table [id=c73df3c3c00e496587dd3cd628728ac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:02.574189 485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20896f58de314dcdbd40d3ad506d0407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:02.590942 867 tablet_bootstrap.cc:492] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256: Bootstrap starting.
I20260809 04:56:02.595239 872 tablet_bootstrap.cc:492] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: Bootstrap starting.
I20260809 04:56:02.593531 870 tablet_bootstrap.cc:492] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f: Bootstrap starting.
I20260809 04:56:02.598819 867 tablet_bootstrap.cc:654] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:02.600312 870 tablet_bootstrap.cc:654] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:02.601404 867 log.cc:824] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:02.601555 872 tablet_bootstrap.cc:654] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:02.602599 870 log.cc:824] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:02.604264 872 log.cc:824] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:02.606215 867 tablet_bootstrap.cc:492] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256: No bootstrap required, opened a new log
I20260809 04:56:02.606666 867 ts_tablet_manager.cc:1397] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.013s
I20260809 04:56:02.607784 870 tablet_bootstrap.cc:492] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f: No bootstrap required, opened a new log
I20260809 04:56:02.608152 870 ts_tablet_manager.cc:1397] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260809 04:56:02.610224 872 tablet_bootstrap.cc:492] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: No bootstrap required, opened a new log
I20260809 04:56:02.610617 872 ts_tablet_manager.cc:1397] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.012s
I20260809 04:56:02.620882 870 raft_consensus.cc:348] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } }
I20260809 04:56:02.621583 870 raft_consensus.cc:374] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:02.621833 870 raft_consensus.cc:729] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34f53fb768143e88b356cd137c36b8f, State: Initialized, Role: FOLLOWER
I20260809 04:56:02.622721 870 consensus_queue.cc:260] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } }
I20260809 04:56:02.622879 867 raft_consensus.cc:348] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } }
I20260809 04:56:02.623442 867 raft_consensus.cc:374] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:02.623708 867 raft_consensus.cc:729] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afd73c2e039542929dd0c4d82005c256, State: Initialized, Role: FOLLOWER
I20260809 04:56:02.624743 867 consensus_queue.cc:260] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } }
I20260809 04:56:02.625563 870 ts_tablet_manager.cc:1428] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f: Time spent starting tablet: real 0.017s user 0.002s sys 0.014s
I20260809 04:56:02.627171 603 heartbeater.cc:503] Master 127.29.243.254:37499 was elected leader, sending a full tablet report...
I20260809 04:56:02.627204 872 raft_consensus.cc:348] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } }
I20260809 04:56:02.627872 872 raft_consensus.cc:374] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:02.628230 872 raft_consensus.cc:729] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f7d35d953441fe9566e1f3ca088d89, State: Initialized, Role: FOLLOWER
I20260809 04:56:02.629057 872 consensus_queue.cc:260] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } }
I20260809 04:56:02.630259 867 ts_tablet_manager.cc:1428] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256: Time spent starting tablet: real 0.023s user 0.014s sys 0.006s
I20260809 04:56:02.631093 833 heartbeater.cc:503] Master 127.29.243.254:37499 was elected leader, sending a full tablet report...
I20260809 04:56:02.631801 872 ts_tablet_manager.cc:1428] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: Time spent starting tablet: real 0.021s user 0.014s sys 0.010s
I20260809 04:56:02.635766 880 raft_consensus.cc:482] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:02.636173 880 raft_consensus.cc:504] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } }
I20260809 04:56:02.638423 880 leader_election.cc:288] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c34f53fb768143e88b356cd137c36b8f (127.29.243.193:44961), 39f7d35d953441fe9566e1f3ca088d89 (127.29.243.195:35159)
I20260809 04:56:02.652645 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.638669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:36547 (local address 127.29.243.193:44961)
0809 04:56:02.638961 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:02.638968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:02.642081 (+ 3113us) server_negotiation.cc:407] Connection header received
0809 04:56:02.642321 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:02.642327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:02.642410 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:02.642517 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:02.643990 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.645340 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.649752 (+ 4412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.651385 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.651603 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.651688 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:02.652171 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 04:56:02.652307 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:56:02.653149 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.639834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:44961 (local address 127.29.243.194:36547)
0809 04:56:02.641987 (+ 2153us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:02.642007 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:02.642187 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:02.642552 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:02.642557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:02.642589 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:02.643834 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.643855 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.645480 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.645486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.649557 (+ 4071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.649578 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.651811 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.651818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.651984 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.652010 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:56:02.652100 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 04:56:02.652241 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1953,"mutex_wait_us":53,"thread_start_us":1377,"threads_started":1}
I20260809 04:56:02.653460 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.639125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:48395 (local address 127.29.243.195:35159)
0809 04:56:02.639371 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:02.639377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:02.643306 (+ 3929us) server_negotiation.cc:407] Connection header received
0809 04:56:02.643485 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:02.643490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:02.643578 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:02.643682 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:02.644890 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.646156 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.650203 (+ 4047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.651993 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.652234 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.652308 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:02.652847 (+ 539us) server_negotiation.cc:299] Negotiation successful
0809 04:56:02.653108 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:56:02.654394 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.640042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:35159 (local address 127.29.243.194:48395)
0809 04:56:02.643149 (+ 3107us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:02.643166 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:02.643344 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:02.643733 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:02.643738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:02.643777 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:02.644687 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.644704 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.646350 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.646354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.650047 (+ 3693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:02.650059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.652362 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.652370 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:02.652633 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.652668 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 04:56:02.653650 (+ 982us) client_negotiation.cc:240] Negotiation successful
0809 04:56:02.653756 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2862,"mutex_wait_us":5,"thread_start_us":2499,"threads_started":1}
I20260809 04:56:02.654448 300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20896f58de314dcdbd40d3ad506d0407" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34f53fb768143e88b356cd137c36b8f" is_pre_election: true
I20260809 04:56:02.655439 300 raft_consensus.cc:2454] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 0.
I20260809 04:56:02.655905 756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20896f58de314dcdbd40d3ad506d0407" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f7d35d953441fe9566e1f3ca088d89" is_pre_election: true
I20260809 04:56:02.656446 409 leader_election.cc:302] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afd73c2e039542929dd0c4d82005c256, c34f53fb768143e88b356cd137c36b8f; no voters:
I20260809 04:56:02.656899 756 raft_consensus.cc:2454] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 0.
I20260809 04:56:02.657171 880 raft_consensus.cc:2781] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:02.657401 880 raft_consensus.cc:482] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:02.657581 880 raft_consensus.cc:3037] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:02.662698 880 raft_consensus.cc:504] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } }
I20260809 04:56:02.663756 880 leader_election.cc:288] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 election: Requested vote from peers c34f53fb768143e88b356cd137c36b8f (127.29.243.193:44961), 39f7d35d953441fe9566e1f3ca088d89 (127.29.243.195:35159)
I20260809 04:56:02.664366 300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20896f58de314dcdbd40d3ad506d0407" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34f53fb768143e88b356cd137c36b8f"
I20260809 04:56:02.664412 756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20896f58de314dcdbd40d3ad506d0407" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f7d35d953441fe9566e1f3ca088d89"
I20260809 04:56:02.664650 300 raft_consensus.cc:3037] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:02.664700 756 raft_consensus.cc:3037] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:02.668985 756 raft_consensus.cc:2454] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 1.
I20260809 04:56:02.669582 410 leader_election.cc:302] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f7d35d953441fe9566e1f3ca088d89, afd73c2e039542929dd0c4d82005c256; no voters:
I20260809 04:56:02.670046 880 raft_consensus.cc:2781] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:02.669955 300 raft_consensus.cc:2454] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 1.
I20260809 04:56:02.671340 880 raft_consensus.cc:686] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 1 LEADER]: Becoming Leader. State: Replica: afd73c2e039542929dd0c4d82005c256, State: Running, Role: LEADER
I20260809 04:56:02.672116 880 consensus_queue.cc:237] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } }
W20260809 04:56:02.674945 838 tablet.cc:2369] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:56:02.678894 32508 catalog_manager.cc:5495] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 reported cstate change: term changed from 0 to 1, leader changed from <none> to afd73c2e039542929dd0c4d82005c256 (127.29.243.194). New cstate: current_term: 1 leader_uuid: "afd73c2e039542929dd0c4d82005c256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } health_report { overall_health: HEALTHY } } }
I20260809 04:56:02.725241 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:02.717817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46314 (local address 127.29.243.194:39139)
0809 04:56:02.718082 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:02.718089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:02.718431 (+ 342us) server_negotiation.cc:407] Connection header received
0809 04:56:02.718729 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:02.718733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:02.718815 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:02.718934 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:02.719528 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.720529 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.722381 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:02.723056 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:02.723153 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:02.723995 (+ 842us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:02.724579 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:02.724724 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:02.724766 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 04:56:02.724836 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 04:56:02.739339 756 raft_consensus.cc:1261] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89 [term 1 FOLLOWER]: Refusing update from remote peer afd73c2e039542929dd0c4d82005c256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:02.741108 884 consensus_queue.cc:1035] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 04:56:02.741706 300 raft_consensus.cc:1261] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f [term 1 FOLLOWER]: Refusing update from remote peer afd73c2e039542929dd0c4d82005c256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:02.742673 884 consensus_queue.cc:1035] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:02.788216 32509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:02.788712 32509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:02.793835 32509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d: Sending DeleteTablet for 3 replicas of tablet 20896f58de314dcdbd40d3ad506d0407
I20260809 04:56:02.795472 715 tablet_service.cc:1514] Processing DeleteTablet for tablet 20896f58de314dcdbd40d3ad506d0407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:02 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:50946
I20260809 04:56:02.795608 485 tablet_service.cc:1514] Processing DeleteTablet for tablet 20896f58de314dcdbd40d3ad506d0407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:02 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:46300
I20260809 04:56:02.797848 32736 tablet_service.cc:1514] Processing DeleteTablet for tablet 20896f58de314dcdbd40d3ad506d0407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:02 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:59156
I20260809 04:56:02.801744 32509 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:02.802903 32509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:02.808415 907 tablet_replica.cc:330] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: stopping tablet replica
I20260809 04:56:02.809083 908 tablet_replica.cc:330] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f: stopping tablet replica
I20260809 04:56:02.810037 908 raft_consensus.cc:2229] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:02.810242 907 raft_consensus.cc:2229] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:02.810904 908 raft_consensus.cc:2258] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:02.812543 907 raft_consensus.cc:2258] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:02.813405 906 tablet_replica.cc:330] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256: stopping tablet replica
I20260809 04:56:02.813673 908 ts_tablet_manager.cc:1905] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:02.814986 906 raft_consensus.cc:2229] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:02.815081 485 tablet_service.cc:1467] Processing CreateTablet for tablet df32e492a1674359bdb9e56d2c5cd2a5 (DEFAULT_TABLE table=test-table [id=94bd0a5faef34aa28858682d0162ef6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:02.815836 485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df32e492a1674359bdb9e56d2c5cd2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:02.816215 715 tablet_service.cc:1467] Processing CreateTablet for tablet df32e492a1674359bdb9e56d2c5cd2a5 (DEFAULT_TABLE table=test-table [id=94bd0a5faef34aa28858682d0162ef6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:02.816624 907 ts_tablet_manager.cc:1905] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:02.816757 715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df32e492a1674359bdb9e56d2c5cd2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:02.817827 906 raft_consensus.cc:2258] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:02.818681 32736 tablet_service.cc:1467] Processing CreateTablet for tablet df32e492a1674359bdb9e56d2c5cd2a5 (DEFAULT_TABLE table=test-table [id=94bd0a5faef34aa28858682d0162ef6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:02.819239 32736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df32e492a1674359bdb9e56d2c5cd2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:02.821893 906 ts_tablet_manager.cc:1905] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:02.828815 908 ts_tablet_manager.cc:1918] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:02.829188 908 log.cc:1196] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/20896f58de314dcdbd40d3ad506d0407
I20260809 04:56:02.830200 908 ts_tablet_manager.cc:1939] T 20896f58de314dcdbd40d3ad506d0407 P c34f53fb768143e88b356cd137c36b8f: Deleting consensus metadata
I20260809 04:56:02.832723 32492 catalog_manager.cc:4841] TS c34f53fb768143e88b356cd137c36b8f (127.29.243.193:44961): tablet 20896f58de314dcdbd40d3ad506d0407 (table test-table [id=c73df3c3c00e496587dd3cd628728ac0]) successfully deleted
I20260809 04:56:02.834596 870 tablet_bootstrap.cc:492] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f: Bootstrap starting.
I20260809 04:56:02.835119 872 tablet_bootstrap.cc:492] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89: Bootstrap starting.
I20260809 04:56:02.837580 870 tablet_bootstrap.cc:654] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:02.838157 872 tablet_bootstrap.cc:654] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:02.845582 867 tablet_bootstrap.cc:492] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256: Bootstrap starting.
I20260809 04:56:02.848958 867 tablet_bootstrap.cc:654] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:02.849613 870 tablet_bootstrap.cc:492] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f: No bootstrap required, opened a new log
I20260809 04:56:02.849606 872 tablet_bootstrap.cc:492] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89: No bootstrap required, opened a new log
I20260809 04:56:02.849808 870 ts_tablet_manager.cc:1397] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260809 04:56:02.849838 872 ts_tablet_manager.cc:1397] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.002s
I20260809 04:56:02.851312 870 raft_consensus.cc:348] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:02.851647 870 raft_consensus.cc:374] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:02.851837 907 ts_tablet_manager.cc:1918] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:02.851882 870 raft_consensus.cc:729] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34f53fb768143e88b356cd137c36b8f, State: Initialized, Role: FOLLOWER
I20260809 04:56:02.852197 907 log.cc:1196] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/20896f58de314dcdbd40d3ad506d0407
I20260809 04:56:02.852427 870 consensus_queue.cc:260] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:02.852452 872 raft_consensus.cc:348] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:02.852773 872 raft_consensus.cc:374] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:02.853003 872 raft_consensus.cc:729] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f7d35d953441fe9566e1f3ca088d89, State: Initialized, Role: FOLLOWER
I20260809 04:56:02.853506 907 ts_tablet_manager.cc:1939] T 20896f58de314dcdbd40d3ad506d0407 P 39f7d35d953441fe9566e1f3ca088d89: Deleting consensus metadata
I20260809 04:56:02.856139 870 ts_tablet_manager.cc:1428] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260809 04:56:02.856494 32493 catalog_manager.cc:4841] TS 39f7d35d953441fe9566e1f3ca088d89 (127.29.243.195:35159): tablet 20896f58de314dcdbd40d3ad506d0407 (table test-table [id=c73df3c3c00e496587dd3cd628728ac0]) successfully deleted
I20260809 04:56:02.853857 872 consensus_queue.cc:260] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:02.857268 872 ts_tablet_manager.cc:1428] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260809 04:56:02.859735 906 ts_tablet_manager.cc:1918] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:02.861182 906 log.cc:1196] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/20896f58de314dcdbd40d3ad506d0407
I20260809 04:56:02.862783 906 ts_tablet_manager.cc:1939] T 20896f58de314dcdbd40d3ad506d0407 P afd73c2e039542929dd0c4d82005c256: Deleting consensus metadata
I20260809 04:56:02.864329 867 tablet_bootstrap.cc:492] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256: No bootstrap required, opened a new log
I20260809 04:56:02.864656 867 ts_tablet_manager.cc:1397] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.009s
I20260809 04:56:02.865084 32492 catalog_manager.cc:4841] TS afd73c2e039542929dd0c4d82005c256 (127.29.243.194:39139): tablet 20896f58de314dcdbd40d3ad506d0407 (table test-table [id=c73df3c3c00e496587dd3cd628728ac0]) successfully deleted
I20260809 04:56:02.866819 867 raft_consensus.cc:348] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:02.867188 867 raft_consensus.cc:374] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:02.867440 867 raft_consensus.cc:729] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afd73c2e039542929dd0c4d82005c256, State: Initialized, Role: FOLLOWER
I20260809 04:56:02.867937 867 consensus_queue.cc:260] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:02.868830 867 ts_tablet_manager.cc:1428] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
W20260809 04:56:03.054895 377 tablet.cc:2369] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:56:03.088542 604 tablet.cc:2369] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:56:03.150916 901 raft_consensus.cc:482] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:03.151216 901 raft_consensus.cc:504] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.152104 901 leader_election.cc:288] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c34f53fb768143e88b356cd137c36b8f (127.29.243.193:44961), 39f7d35d953441fe9566e1f3ca088d89 (127.29.243.195:35159)
I20260809 04:56:03.152709 756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df32e492a1674359bdb9e56d2c5cd2a5" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f7d35d953441fe9566e1f3ca088d89" is_pre_election: true
I20260809 04:56:03.153007 756 raft_consensus.cc:2454] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 0.
I20260809 04:56:03.152966 300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df32e492a1674359bdb9e56d2c5cd2a5" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34f53fb768143e88b356cd137c36b8f" is_pre_election: true
I20260809 04:56:03.153272 300 raft_consensus.cc:2454] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 0.
I20260809 04:56:03.153494 410 leader_election.cc:302] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f7d35d953441fe9566e1f3ca088d89, afd73c2e039542929dd0c4d82005c256; no voters:
I20260809 04:56:03.153945 901 raft_consensus.cc:2781] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:03.154072 901 raft_consensus.cc:482] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:03.154171 901 raft_consensus.cc:3037] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:03.158221 901 raft_consensus.cc:504] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.159036 901 leader_election.cc:288] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 election: Requested vote from peers c34f53fb768143e88b356cd137c36b8f (127.29.243.193:44961), 39f7d35d953441fe9566e1f3ca088d89 (127.29.243.195:35159)
I20260809 04:56:03.159564 756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df32e492a1674359bdb9e56d2c5cd2a5" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f7d35d953441fe9566e1f3ca088d89"
I20260809 04:56:03.159775 756 raft_consensus.cc:3037] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:03.159672 300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df32e492a1674359bdb9e56d2c5cd2a5" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34f53fb768143e88b356cd137c36b8f"
I20260809 04:56:03.159974 300 raft_consensus.cc:3037] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:03.162361 756 raft_consensus.cc:2454] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 1.
I20260809 04:56:03.162972 410 leader_election.cc:302] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f7d35d953441fe9566e1f3ca088d89, afd73c2e039542929dd0c4d82005c256; no voters:
I20260809 04:56:03.163302 901 raft_consensus.cc:2781] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:03.163483 901 raft_consensus.cc:686] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 1 LEADER]: Becoming Leader. State: Replica: afd73c2e039542929dd0c4d82005c256, State: Running, Role: LEADER
I20260809 04:56:03.163640 300 raft_consensus.cc:2454] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 1.
I20260809 04:56:03.163864 901 consensus_queue.cc:237] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.167459 32509 catalog_manager.cc:5495] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 reported cstate change: term changed from 0 to 1, leader changed from <none> to afd73c2e039542929dd0c4d82005c256 (127.29.243.194). New cstate: current_term: 1 leader_uuid: "afd73c2e039542929dd0c4d82005c256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } health_report { overall_health: UNKNOWN } } }
Aug 09 04:56:03 dist-test-slave-21b9 krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251360, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:03.249653 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:03.230116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37499 (local address 127.0.0.1:53150)
0809 04:56:03.231474 (+ 1358us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:03.231492 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:03.231671 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:03.232000 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:03.232004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:03.233030 (+ 1026us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:03.233417 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:03.233430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.234876 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.234880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:03.236355 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:03.236368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.237112 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.237118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:03.237323 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:03.238259 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:03.238277 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:03.241720 (+ 3443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:03.246315 (+ 4595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:03.246330 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:03.246339 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:03.247046 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:03.247522 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:03.247528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:03.247532 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:03.247738 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:03.248314 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:03.248323 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:03.248491 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 04:56:03.248660 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 04:56:03.248934 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1198,"thread_start_us":947,"threads_started":1}
I20260809 04:56:03.249735 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:03.230100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.29.243.254:37499)
0809 04:56:03.230935 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:03.230943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:03.231584 (+ 641us) server_negotiation.cc:407] Connection header received
0809 04:56:03.231792 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:03.231796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:03.231853 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:03.231942 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:03.233593 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.234735 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.236488 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.236953 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.237064 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:03.241949 (+ 4885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:03.241954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:03.241957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:03.241987 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:03.246189 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.247163 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.247167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.247169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.247419 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.247842 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.247848 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.247851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.248178 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:03.248353 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:03.248781 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 04:56:03.249081 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":650,"thread_start_us":407,"threads_started":1}
I20260809 04:56:03.257309 32509 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:53150:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:56:03.258678 32509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:03.266461 32736 tablet_service.cc:1467] Processing CreateTablet for tablet f80f2566d1414e85aa87e0e5c30a8f3c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df971ca057c745189a958cc3d745300e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:56:03.266851 715 tablet_service.cc:1467] Processing CreateTablet for tablet f80f2566d1414e85aa87e0e5c30a8f3c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df971ca057c745189a958cc3d745300e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:56:03.267174 32736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f80f2566d1414e85aa87e0e5c30a8f3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:03.267582 715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f80f2566d1414e85aa87e0e5c30a8f3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:03.267978 485 tablet_service.cc:1467] Processing CreateTablet for tablet f80f2566d1414e85aa87e0e5c30a8f3c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df971ca057c745189a958cc3d745300e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:56:03.268916 485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f80f2566d1414e85aa87e0e5c30a8f3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:03.276861 870 tablet_bootstrap.cc:492] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f: Bootstrap starting.
I20260809 04:56:03.278612 872 tablet_bootstrap.cc:492] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89: Bootstrap starting.
I20260809 04:56:03.279603 867 tablet_bootstrap.cc:492] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: Bootstrap starting.
I20260809 04:56:03.280009 870 tablet_bootstrap.cc:654] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:03.281641 872 tablet_bootstrap.cc:654] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:03.282512 867 tablet_bootstrap.cc:654] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:03.291090 872 tablet_bootstrap.cc:492] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89: No bootstrap required, opened a new log
I20260809 04:56:03.291381 872 ts_tablet_manager.cc:1397] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.002s
I20260809 04:56:03.292213 867 tablet_bootstrap.cc:492] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: No bootstrap required, opened a new log
I20260809 04:56:03.292543 872 raft_consensus.cc:348] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.292822 867 ts_tablet_manager.cc:1397] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.010s
I20260809 04:56:03.292987 872 raft_consensus.cc:374] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:03.293221 872 raft_consensus.cc:729] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f7d35d953441fe9566e1f3ca088d89, State: Initialized, Role: FOLLOWER
I20260809 04:56:03.293646 872 consensus_queue.cc:260] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.294535 872 ts_tablet_manager.cc:1428] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 04:56:03.294723 870 tablet_bootstrap.cc:492] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f: No bootstrap required, opened a new log
I20260809 04:56:03.294984 870 ts_tablet_manager.cc:1397] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.004s
I20260809 04:56:03.294543 881 tablet_replica.cc:439] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } }
I20260809 04:56:03.295878 881 tablet_replica.cc:442] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:03.295987 870 raft_consensus.cc:348] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.296303 870 raft_consensus.cc:374] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:03.296487 870 raft_consensus.cc:729] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34f53fb768143e88b356cd137c36b8f, State: Initialized, Role: FOLLOWER
I20260809 04:56:03.296543 867 raft_consensus.cc:348] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.296893 870 consensus_queue.cc:260] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.297629 878 tablet_replica.cc:439] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } }
I20260809 04:56:03.298182 878 tablet_replica.cc:442] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:03.299432 870 ts_tablet_manager.cc:1428] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 04:56:03.297075 867 raft_consensus.cc:374] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:03.302337 867 raft_consensus.cc:729] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afd73c2e039542929dd0c4d82005c256, State: Initialized, Role: FOLLOWER
I20260809 04:56:03.302764 867 consensus_queue.cc:260] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.303531 901 tablet_replica.cc:439] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } }
I20260809 04:56:03.304085 867 ts_tablet_manager.cc:1428] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260809 04:56:03.304106 901 tablet_replica.cc:442] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:03.323969 901 raft_consensus.cc:482] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:03.324335 901 raft_consensus.cc:504] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.325304 901 leader_election.cc:288] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c34f53fb768143e88b356cd137c36b8f (127.29.243.193:44961), 39f7d35d953441fe9566e1f3ca088d89 (127.29.243.195:35159)
I20260809 04:56:03.325897 300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80f2566d1414e85aa87e0e5c30a8f3c" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34f53fb768143e88b356cd137c36b8f" is_pre_election: true
I20260809 04:56:03.326071 756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80f2566d1414e85aa87e0e5c30a8f3c" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f7d35d953441fe9566e1f3ca088d89" is_pre_election: true
I20260809 04:56:03.326305 300 raft_consensus.cc:2454] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 0.
I20260809 04:56:03.326428 756 raft_consensus.cc:2454] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 0.
I20260809 04:56:03.326978 410 leader_election.cc:302] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f7d35d953441fe9566e1f3ca088d89, afd73c2e039542929dd0c4d82005c256; no voters:
I20260809 04:56:03.327381 901 raft_consensus.cc:2781] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:03.327565 901 raft_consensus.cc:482] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:03.327797 901 raft_consensus.cc:3037] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:03.331147 901 raft_consensus.cc:504] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.332142 901 leader_election.cc:288] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 election: Requested vote from peers c34f53fb768143e88b356cd137c36b8f (127.29.243.193:44961), 39f7d35d953441fe9566e1f3ca088d89 (127.29.243.195:35159)
I20260809 04:56:03.332484 300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80f2566d1414e85aa87e0e5c30a8f3c" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34f53fb768143e88b356cd137c36b8f"
I20260809 04:56:03.332742 756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80f2566d1414e85aa87e0e5c30a8f3c" candidate_uuid: "afd73c2e039542929dd0c4d82005c256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f7d35d953441fe9566e1f3ca088d89"
I20260809 04:56:03.332863 300 raft_consensus.cc:3037] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:03.333112 756 raft_consensus.cc:3037] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:03.336429 756 raft_consensus.cc:2454] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 1.
I20260809 04:56:03.336670 300 raft_consensus.cc:2454] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afd73c2e039542929dd0c4d82005c256 in term 1.
I20260809 04:56:03.337066 410 leader_election.cc:302] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f7d35d953441fe9566e1f3ca088d89, afd73c2e039542929dd0c4d82005c256; no voters:
I20260809 04:56:03.337378 901 raft_consensus.cc:2781] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:03.337579 901 raft_consensus.cc:686] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [term 1 LEADER]: Becoming Leader. State: Replica: afd73c2e039542929dd0c4d82005c256, State: Running, Role: LEADER
I20260809 04:56:03.337867 901 consensus_queue.cc:237] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } }
I20260809 04:56:03.340543 880 tablet_replica.cc:439] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: TxnStatusTablet state changed. Reason: New leader afd73c2e039542929dd0c4d82005c256. Latest consensus state: current_term: 1 leader_uuid: "afd73c2e039542929dd0c4d82005c256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } }
I20260809 04:56:03.341470 880 tablet_replica.cc:442] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:56:03.342988 942 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:56:03.342162 32507 catalog_manager.cc:5495] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 reported cstate change: term changed from 0 to 1, leader changed from <none> to afd73c2e039542929dd0c4d82005c256 (127.29.243.194). New cstate: current_term: 1 leader_uuid: "afd73c2e039542929dd0c4d82005c256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } health_report { overall_health: UNKNOWN } } }
I20260809 04:56:03.343497 942 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:56:03.367545 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:03.358464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:39139 (local address 127.0.0.1:46330)
0809 04:56:03.358737 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:03.358759 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:03.358923 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:03.360608 (+ 1685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:03.360613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:03.360625 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:56:03.360998 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:03.361010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.362791 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.362797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:03.364552 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:03.364566 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.365452 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.365459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:03.365874 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:03.365967 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:56:03.366519 (+ 552us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:56:03.366541 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:56:03.366610 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 04:56:03.366809 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 04:56:03.367751 944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:03.359084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46330 (local address 127.29.243.194:39139)
0809 04:56:03.360319 (+ 1235us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:03.360324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:03.360340 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:56:03.360393 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:03.360398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:03.360465 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:03.360552 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:03.361170 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.362657 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.364687 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.365265 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.365372 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:03.366291 (+ 919us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:03.366412 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:03.366495 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:03.366790 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 04:56:03.366869 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1120,"thread_start_us":773,"threads_started":1}
I20260809 04:56:03.373747 756 raft_consensus.cc:1261] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89 [term 1 FOLLOWER]: Refusing update from remote peer afd73c2e039542929dd0c4d82005c256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:03.373878 300 raft_consensus.cc:1261] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f [term 1 FOLLOWER]: Refusing update from remote peer afd73c2e039542929dd0c4d82005c256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:03.374475 880 consensus_queue.cc:1035] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:03.375044 901 consensus_queue.cc:1035] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:03.384807 881 tablet_replica.cc:439] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89: TxnStatusTablet state changed. Reason: New leader afd73c2e039542929dd0c4d82005c256. Latest consensus state: current_term: 1 leader_uuid: "afd73c2e039542929dd0c4d82005c256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } }
I20260809 04:56:03.385267 881 tablet_replica.cc:442] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:03.389603 948 mvcc.cc:205] Tried to move back new op lower bound from 7316485584372084736 to 7316485584234233856. Current Snapshot: MvccSnapshot[applied={T|T < 7316485584372084736}]
I20260809 04:56:03.391453 901 tablet_replica.cc:439] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afd73c2e039542929dd0c4d82005c256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } }
I20260809 04:56:03.391871 901 tablet_replica.cc:442] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:56:03.400988 881 tablet_replica.cc:439] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afd73c2e039542929dd0c4d82005c256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } }
I20260809 04:56:03.401336 881 tablet_replica.cc:442] T f80f2566d1414e85aa87e0e5c30a8f3c P 39f7d35d953441fe9566e1f3ca088d89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:03.402864 878 tablet_replica.cc:439] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f: TxnStatusTablet state changed. Reason: New leader afd73c2e039542929dd0c4d82005c256. Latest consensus state: current_term: 1 leader_uuid: "afd73c2e039542929dd0c4d82005c256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } }
I20260809 04:56:03.403447 901 tablet_replica.cc:439] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afd73c2e039542929dd0c4d82005c256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } }
I20260809 04:56:03.404083 901 tablet_replica.cc:442] T f80f2566d1414e85aa87e0e5c30a8f3c P afd73c2e039542929dd0c4d82005c256: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:56:03.406284 958 tablet_replica.cc:439] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afd73c2e039542929dd0c4d82005c256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 } } peers { permanent_uuid: "afd73c2e039542929dd0c4d82005c256" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 39139 } } peers { permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 } } }
I20260809 04:56:03.406841 958 tablet_replica.cc:442] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:03.408048 878 tablet_replica.cc:442] T f80f2566d1414e85aa87e0e5c30a8f3c P c34f53fb768143e88b356cd137c36b8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:03.437674 944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:03.425289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:41461 (local address 127.29.243.194:39139)
0809 04:56:03.425463 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:03.425469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:03.427041 (+ 1572us) server_negotiation.cc:407] Connection header received
0809 04:56:03.427384 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:03.427390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:03.427486 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:03.427609 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:03.429046 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.431191 (+ 2145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.434587 (+ 3396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.435596 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.435744 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:03.436548 (+ 804us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:03.436694 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:03.436803 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:03.436982 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 04:56:03.437102 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":59}
I20260809 04:56:03.438297 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:03.425434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:39139 (local address 127.29.243.194:41461)
0809 04:56:03.426929 (+ 1495us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:03.426951 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:03.427192 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:03.427672 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:03.427677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:03.427692 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:56:03.428581 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:03.428596 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.431385 (+ 2789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.431391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:03.434406 (+ 3015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:03.434422 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.435920 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.435929 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:03.436220 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:03.436303 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:56:03.436818 (+ 515us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:56:03.436832 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 04:56:03.436925 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 04:56:03.437135 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1195,"mutex_wait_us":39,"thread_start_us":896,"threads_started":1}
I20260809 04:56:03.451726 300 raft_consensus.cc:1261] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f [term 1 FOLLOWER]: Refusing update from remote peer afd73c2e039542929dd0c4d82005c256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:03.451781 756 raft_consensus.cc:1261] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89 [term 1 FOLLOWER]: Refusing update from remote peer afd73c2e039542929dd0c4d82005c256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:03.452527 901 consensus_queue.cc:1035] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c34f53fb768143e88b356cd137c36b8f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 44961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:03.452991 880 consensus_queue.cc:1035] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f7d35d953441fe9566e1f3ca088d89" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 35159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:03.538723 32508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:03.539108 32508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:03.542866 32508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd82eae58d2145698a80ac43b6f4e15d: Sending DeleteTablet for 3 replicas of tablet df32e492a1674359bdb9e56d2c5cd2a5
I20260809 04:56:03.543972 485 tablet_service.cc:1514] Processing DeleteTablet for tablet df32e492a1674359bdb9e56d2c5cd2a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:03 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:46300
I20260809 04:56:03.544008 32736 tablet_service.cc:1514] Processing DeleteTablet for tablet df32e492a1674359bdb9e56d2c5cd2a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:03 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:59156
I20260809 04:56:03.544076 715 tablet_service.cc:1514] Processing DeleteTablet for tablet df32e492a1674359bdb9e56d2c5cd2a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:03 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:50946
I20260809 04:56:03.557605 972 tablet_replica.cc:330] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f: stopping tablet replica
I20260809 04:56:03.558077 972 raft_consensus.cc:2229] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:03.558753 972 raft_consensus.cc:2258] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:03.560281 972 ts_tablet_manager.cc:1905] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:03.560765 974 tablet_replica.cc:330] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89: stopping tablet replica
I20260809 04:56:03.561090 974 raft_consensus.cc:2229] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:03.561662 974 raft_consensus.cc:2258] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:03.562943 974 ts_tablet_manager.cc:1905] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:03.570510 973 tablet_replica.cc:330] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256: stopping tablet replica
I20260809 04:56:03.570890 973 raft_consensus.cc:2229] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:03.571766 973 raft_consensus.cc:2258] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:03.573246 973 ts_tablet_manager.cc:1905] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:03.578006 972 ts_tablet_manager.cc:1918] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:03.578023 974 ts_tablet_manager.cc:1918] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:03.578253 974 log.cc:1196] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/df32e492a1674359bdb9e56d2c5cd2a5
I20260809 04:56:03.578253 972 log.cc:1196] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/df32e492a1674359bdb9e56d2c5cd2a5
I20260809 04:56:03.579025 974 ts_tablet_manager.cc:1939] T df32e492a1674359bdb9e56d2c5cd2a5 P 39f7d35d953441fe9566e1f3ca088d89: Deleting consensus metadata
I20260809 04:56:03.579028 972 ts_tablet_manager.cc:1939] T df32e492a1674359bdb9e56d2c5cd2a5 P c34f53fb768143e88b356cd137c36b8f: Deleting consensus metadata
I20260809 04:56:03.580276 32493 catalog_manager.cc:4841] TS 39f7d35d953441fe9566e1f3ca088d89 (127.29.243.195:35159): tablet df32e492a1674359bdb9e56d2c5cd2a5 (table test-table [id=94bd0a5faef34aa28858682d0162ef6d]) successfully deleted
I20260809 04:56:03.581314 32492 catalog_manager.cc:4841] TS c34f53fb768143e88b356cd137c36b8f (127.29.243.193:44961): tablet df32e492a1674359bdb9e56d2c5cd2a5 (table test-table [id=94bd0a5faef34aa28858682d0162ef6d]) successfully deleted
I20260809 04:56:03.587693 973 ts_tablet_manager.cc:1918] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:03.588064 973 log.cc:1196] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/df32e492a1674359bdb9e56d2c5cd2a5
I20260809 04:56:03.589041 973 ts_tablet_manager.cc:1939] T df32e492a1674359bdb9e56d2c5cd2a5 P afd73c2e039542929dd0c4d82005c256: Deleting consensus metadata
I20260809 04:56:03.590500 32492 catalog_manager.cc:4841] TS afd73c2e039542929dd0c4d82005c256 (127.29.243.194:39139): tablet df32e492a1674359bdb9e56d2c5cd2a5 (table test-table [id=94bd0a5faef34aa28858682d0162ef6d]) successfully deleted
Aug 09 04:56:03 dist-test-slave-21b9 krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251362, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:56:03.603075 979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:03.563100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59164 (local address 127.29.243.193:44961)
0809 04:56:03.575352 (+ 12252us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:03.575359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:03.575384 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:56:03.575451 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:03.575456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:03.575548 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:03.575667 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:03.579926 (+ 4259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.581630 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.583656 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.584438 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.584606 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:03.594163 (+ 9557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:03.594181 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:03.594198 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:03.594234 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:03.598863 (+ 4629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.600246 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.600255 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.600257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.600485 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.600953 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.600957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.600960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.601570 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:03.601940 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:03.602240 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 04:56:03.602479 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8009,"thread_start_us":7802,"threads_started":1}
W20260809 04:56:03.604467 368 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59164
I20260809 04:56:03.630252 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:03.608314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.29.243.254:37499)
0809 04:56:03.608890 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:03.608894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:03.608912 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:56:03.609034 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:03.609038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:03.609110 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:03.609187 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:03.612051 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.613140 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.616105 (+ 2965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.616848 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.616956 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:03.621836 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:03.621843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:03.621846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:03.621881 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:03.625915 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.627017 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.627022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.627025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.627221 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.627762 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.627766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.627768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.628239 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:03.628559 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:03.629016 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 04:56:03.629788 (+ 772us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422}
W20260809 04:56:03.631510 32508 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53162
I20260809 04:56:03.634683 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32630
I20260809 04:56:03.653614 979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:03.635653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59174 (local address 127.29.243.193:44961)
0809 04:56:03.635880 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:03.635886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:03.636262 (+ 376us) server_negotiation.cc:407] Connection header received
0809 04:56:03.636470 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:03.636476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:03.636565 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:03.636681 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:03.637820 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.638988 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.640294 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.641092 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.641242 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:03.644748 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:03.644755 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:03.644760 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:03.644803 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:03.649786 (+ 4983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.650646 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.650653 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.650656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.650881 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.651413 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.651418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.651422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.651948 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:03.652234 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:03.652732 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 04:56:03.653016 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 04:56:03.654407 368 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59174
W20260809 04:56:03.655205 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32630: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:56:03.655578 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 32630
I20260809 04:56:03.686257 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 381
Aug 09 04:56:03 dist-test-slave-21b9 krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251362, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:56:03.702500 944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:03.687006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46344 (local address 127.29.243.194:39139)
0809 04:56:03.687219 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:03.687225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:03.687342 (+ 117us) server_negotiation.cc:407] Connection header received
0809 04:56:03.687559 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:03.687563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:03.687637 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:03.687738 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:03.688987 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.690284 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.691654 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.692100 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.692212 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:03.696430 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:03.696445 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:03.696464 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:03.696498 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:03.699529 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.700367 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.700375 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.700377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.700519 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.700943 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.700947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.700950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.701333 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:03.701670 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:03.702005 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 04:56:03.702174 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 04:56:03.703233 592 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46344
W20260809 04:56:03.704038 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 381: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:56:03.704214 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 381
I20260809 04:56:03.733664 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 608
Aug 09 04:56:03 dist-test-slave-21b9 krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251362, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:56:03.748283 988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:03.734382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.29.243.195:35159)
0809 04:56:03.734860 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:03.734866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:03.734883 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:56:03.734947 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:03.734950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:03.735019 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:03.735111 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:03.736424 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.737197 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.738382 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:03.738809 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:03.738903 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:03.742977 (+ 4074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:03.742991 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:03.743016 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:03.743049 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:03.745724 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.746316 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.746323 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.746326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.746505 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:03.746790 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:03.746794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:03.746796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:03.747213 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:03.747620 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:03.747815 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 04:56:03.748007 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":233,"threads_started":1}
W20260809 04:56:03.748965 826 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50958
W20260809 04:56:03.749571 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 608: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:56:03.749707 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 608
I20260809 04:56:03.777582 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32462
I20260809 04:56:03.778731 32570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:03.894105 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 32462
2026-08-09T04:56:03Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5094 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:56:03 dist-test-slave-21b9 krb5kdc[996](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:56:03 dist-test-slave-21b9 krb5kdc[996](info): set up 2 sockets
Aug 09 04:56:03 dist-test-slave-21b9 krb5kdc[996](info): commencing operation
krb5kdc: starting...
W20260809 04:56:05.103492 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.006s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:56:05 dist-test-slave-21b9 krb5kdc[996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:56:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:56:05.304771 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35799
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:35799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:05.638360 1014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:05.638659 1014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:05.638792 1014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:05.646075 1014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:05.646174 1014 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:56:05.646239 1014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:05.646338 1014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:05.646400 1014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:05.646497 1014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:05.656263 1014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:35799
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:05.657536 1014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:05.658855 1014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:05.670428 1026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:05.670388 1027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:05.670473 1030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:05.671303 1014 server_base.cc:1034] running on GCE node
I20260809 04:56:05.672034 1014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:05.673400 1014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:05.674623 1014 hybrid_clock.cc:648] HybridClock initialized: now 1786251365674551 us; error 48 us; skew 500 ppm
I20260809 04:56:05.674970 1014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:05 dist-test-slave-21b9 krb5kdc[996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251365, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:05.678105 1014 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:56:05.679536 1014 webserver.cc:460] Webserver started at http://127.29.243.254:40053/ using document root <none> and password file <none>
I20260809 04:56:05.680226 1014 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:05.680327 1014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:05.680624 1014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:05.683652 1014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "17726e5a5ac04c9b81e0873424d842c5"
format_stamp: "Formatted at 2026-08-09 04:56:05 on dist-test-slave-21b9"
server_key: "04c8dee72b0ffd5c333034cd73264652"
server_key_iv: "ae1b923d89477a876d5a42083f63e132"
server_key_version: "encryptionkey@0"
I20260809 04:56:05.684424 1014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "17726e5a5ac04c9b81e0873424d842c5"
format_stamp: "Formatted at 2026-08-09 04:56:05 on dist-test-slave-21b9"
server_key: "04c8dee72b0ffd5c333034cd73264652"
server_key_iv: "ae1b923d89477a876d5a42083f63e132"
server_key_version: "encryptionkey@0"
I20260809 04:56:05.689662 1014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 04:56:05.693297 1041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:05.694662 1014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:05.694841 1014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "17726e5a5ac04c9b81e0873424d842c5"
format_stamp: "Formatted at 2026-08-09 04:56:05 on dist-test-slave-21b9"
server_key: "04c8dee72b0ffd5c333034cd73264652"
server_key_iv: "ae1b923d89477a876d5a42083f63e132"
server_key_version: "encryptionkey@0"
I20260809 04:56:05.695055 1014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:05.715912 1014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:05.720253 1014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:05.720504 1014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:05.740972 1014 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:35799
I20260809 04:56:05.740976 1152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:35799 every 8 connection(s)
I20260809 04:56:05.742340 1014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:05.743965 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 1014
I20260809 04:56:05.744221 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:05.745079 30671 external_mini_cluster.cc:1428] Setting key 2ee2f4cd0125d776191a1ee7590c6c78
I20260809 04:56:05.747171 1153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:56:05 dist-test-slave-21b9 krb5kdc[996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:05.761096 1153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5: Bootstrap starting.
I20260809 04:56:05.764640 1153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:05.765916 1153 log.cc:824] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:05.768906 1158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:05.747086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.29.243.254:35799)
0809 04:56:05.747999 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:05.748012 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:05.748056 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:56:05.750211 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:05.750234 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:05.750643 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:05.750964 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:05.752800 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:05.754207 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:05.755953 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:05.756629 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:05.756757 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:05.761649 (+ 4892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:05.761657 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:05.761672 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:05.761709 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:05.764726 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:05.765525 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:05.765532 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:05.765538 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:05.765708 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:05.766147 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:05.766150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:05.766152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:05.766865 (+ 713us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:05.767172 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:05.767535 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 04:56:05.767924 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":441,"threads_started":1}
I20260809 04:56:05.769097 1153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5: No bootstrap required, opened a new log
I20260809 04:56:05.776473 1153 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17726e5a5ac04c9b81e0873424d842c5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35799 } }
I20260809 04:56:05.776827 1153 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:05.776919 1153 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17726e5a5ac04c9b81e0873424d842c5, State: Initialized, Role: FOLLOWER
I20260809 04:56:05.777458 1153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17726e5a5ac04c9b81e0873424d842c5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35799 } }
I20260809 04:56:05.777696 1153 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:05.777833 1153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:05.778016 1153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:05.780196 1153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17726e5a5ac04c9b81e0873424d842c5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35799 } }
I20260809 04:56:05.780689 1153 leader_election.cc:302] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17726e5a5ac04c9b81e0873424d842c5; no voters:
I20260809 04:56:05.781273 1153 leader_election.cc:288] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:05.781447 1160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:05.782174 1160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [term 1 LEADER]: Becoming Leader. State: Replica: 17726e5a5ac04c9b81e0873424d842c5, State: Running, Role: LEADER
I20260809 04:56:05.782788 1160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17726e5a5ac04c9b81e0873424d842c5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35799 } }
I20260809 04:56:05.783542 1153 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:05.785776 1161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17726e5a5ac04c9b81e0873424d842c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17726e5a5ac04c9b81e0873424d842c5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35799 } } }
I20260809 04:56:05.785787 1162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17726e5a5ac04c9b81e0873424d842c5. Latest consensus state: current_term: 1 leader_uuid: "17726e5a5ac04c9b81e0873424d842c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17726e5a5ac04c9b81e0873424d842c5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35799 } } }
I20260809 04:56:05.786224 1161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:05.786242 1162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:05.791951 1173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:05.796375 1173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:05.805819 1173 catalog_manager.cc:1347] Generated new cluster ID: 745a03d5af39450585fe2922b6e1ed59
I20260809 04:56:05.805924 1173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:05.827741 1173 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:05.829279 1173 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:05.836225 1173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5: Generated new TSK 0
I20260809 04:56:05.837018 1173 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:56:05.886204 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35799
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:06.231516 1198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:06.231796 1198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:06.231927 1198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:06.238951 1198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:06.239051 1198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:06.239176 1198 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:56:06.239238 1198 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:56:06.239346 1198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:56:06.249080 1198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.243.254:35799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:06.250411 1198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:06.251772 1198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:06.264429 1208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:06.264477 1209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:06.264500 1215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:06.265415 1198 server_base.cc:1034] running on GCE node
I20260809 04:56:06.266006 1198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:06.266896 1198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:06.268095 1198 hybrid_clock.cc:648] HybridClock initialized: now 1786251366268042 us; error 47 us; skew 500 ppm
I20260809 04:56:06.268419 1198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:06 dist-test-slave-21b9 krb5kdc[996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:06.271535 1198 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:56:06.272910 1198 webserver.cc:460] Webserver started at http://127.29.243.193:37367/ using document root <none> and password file <none>
I20260809 04:56:06.273545 1198 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:06.273653 1198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:06.273923 1198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:06.276800 1198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "34a34e4d437d429a84641f1134069a0a"
format_stamp: "Formatted at 2026-08-09 04:56:06 on dist-test-slave-21b9"
server_key: "a69dfb10b01ddf9c7e5f58c1d3dd9c0c"
server_key_iv: "abacf35cceae02d24e0df0784cc7e224"
server_key_version: "encryptionkey@0"
I20260809 04:56:06.277513 1198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "34a34e4d437d429a84641f1134069a0a"
format_stamp: "Formatted at 2026-08-09 04:56:06 on dist-test-slave-21b9"
server_key: "a69dfb10b01ddf9c7e5f58c1d3dd9c0c"
server_key_iv: "abacf35cceae02d24e0df0784cc7e224"
server_key_version: "encryptionkey@0"
I20260809 04:56:06.282600 1198 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 04:56:06.286044 1223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:06.287474 1198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:06.287649 1198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "34a34e4d437d429a84641f1134069a0a"
format_stamp: "Formatted at 2026-08-09 04:56:06 on dist-test-slave-21b9"
server_key: "a69dfb10b01ddf9c7e5f58c1d3dd9c0c"
server_key_iv: "abacf35cceae02d24e0df0784cc7e224"
server_key_version: "encryptionkey@0"
I20260809 04:56:06.287833 1198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:06.305248 1198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:06.309309 1198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:06.309554 1198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:06.312227 1198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:06.312323 1198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:06.312453 1198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:06.312526 1198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:56:06 dist-test-slave-21b9 krb5kdc[996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:06.335580 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:06.312684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:40033 (local address 127.29.243.254:35799)
0809 04:56:06.313454 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:06.313461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:06.314396 (+ 935us) server_negotiation.cc:407] Connection header received
0809 04:56:06.317157 (+ 2761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:06.317163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:06.317254 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:06.317383 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:06.319992 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.321112 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.323321 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.323750 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.323865 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:06.328296 (+ 4431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:06.328303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:06.328308 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:06.328353 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:06.331716 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:06.332680 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:06.332686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:06.332689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:06.332812 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:06.333489 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:06.333494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:06.333496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:06.333806 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:06.333961 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:06.334996 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0809 04:56:06.335218 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":527,"threads_started":1}
I20260809 04:56:06.337127 1246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:06.313238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35799 (local address 127.29.243.193:40033)
0809 04:56:06.314206 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:06.314261 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:06.316797 (+ 2536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:06.317665 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:06.317677 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:06.318580 (+ 903us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:06.319727 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:06.319760 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.321313 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.321319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:06.323133 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:06.323164 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.323991 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.323999 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:06.324210 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:06.324881 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:06.324923 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:06.328060 (+ 3137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:06.331871 (+ 3811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:06.331880 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:06.331897 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:06.332514 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:06.332953 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:06.332974 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:06.332977 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:06.333349 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:06.333963 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:06.333972 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:06.334530 (+ 558us) client_negotiation.cc:769] Sending connection context
0809 04:56:06.334897 (+ 367us) client_negotiation.cc:240] Negotiation successful
0809 04:56:06.335546 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":355,"threads_started":1}
I20260809 04:56:06.357101 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:06.344061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:36205 (local address 127.29.243.254:35799)
0809 04:56:06.344240 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:06.344247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:06.344826 (+ 579us) server_negotiation.cc:407] Connection header received
0809 04:56:06.345071 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:06.345077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:06.345149 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:06.345226 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:06.346162 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.347103 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.348833 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.349259 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.349369 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:06.351068 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:06.351074 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:06.351077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:06.351107 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:06.353700 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:06.354684 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:06.354690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:06.354693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:06.354870 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:06.355291 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:06.355297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:06.355301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:06.355672 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:06.355830 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:06.356476 (+ 646us) server_negotiation.cc:299] Negotiation successful
0809 04:56:06.356722 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:56:06.357296 1359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:06.344103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35799 (local address 127.29.243.193:36205)
0809 04:56:06.344720 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:06.344741 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:06.344948 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:06.345278 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:06.345282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:06.345735 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:06.346022 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:06.346031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.347229 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.347234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:06.348702 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:06.348715 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.349450 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.349457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:06.349593 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:06.350154 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:06.350171 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:06.350898 (+ 727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:06.353829 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:06.353835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:06.353840 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:06.354552 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:06.355010 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:06.355013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:06.355016 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:06.355157 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:06.355915 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:06.355920 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:06.356168 (+ 248us) client_negotiation.cc:769] Sending connection context
0809 04:56:06.356351 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 04:56:06.356718 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":320,"threads_started":1}
I20260809 04:56:06.362139 1198 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:45813
I20260809 04:56:06.362195 1425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:45813 every 8 connection(s)
I20260809 04:56:06.363346 1198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:56:06.365206 1426 heartbeater.cc:347] Connected to a master server at 127.29.243.254:35799
I20260809 04:56:06.365424 1426 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:06.366019 1426 heartbeater.cc:511] Master 127.29.243.254:35799 requested a full tablet report, sending...
I20260809 04:56:06.367960 1065 ts_manager.cc:194] Registered new tserver with Master: 34a34e4d437d429a84641f1134069a0a (127.29.243.193:45813)
I20260809 04:56:06.369014 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 1198
I20260809 04:56:06.369201 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:56:06.369536 1065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:40033
I20260809 04:56:06.369853 30671 external_mini_cluster.cc:1428] Setting key 8cb7d13a9a37f5b6547572ebf9f7b626
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:56:06.414127 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35799
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:06.746357 1432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:06.746639 1432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:06.746771 1432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:06.754060 1432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:06.754168 1432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:06.754288 1432 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:56:06.754349 1432 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:56:06.754458 1432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:56:06.764205 1432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.243.254:35799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:06.765554 1432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:06.766897 1432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:06.780160 1439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:06.780212 1444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:06.780128 1440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:06.780892 1432 server_base.cc:1034] running on GCE node
I20260809 04:56:06.781476 1432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:06.782387 1432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:06.783586 1432 hybrid_clock.cc:648] HybridClock initialized: now 1786251366783519 us; error 59 us; skew 500 ppm
I20260809 04:56:06.783901 1432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:06 dist-test-slave-21b9 krb5kdc[996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:06.786945 1432 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:56:06.788391 1432 webserver.cc:460] Webserver started at http://127.29.243.194:41175/ using document root <none> and password file <none>
I20260809 04:56:06.789111 1432 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:06.789232 1432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:06.789546 1432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:06.792557 1432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "d40828a794a945fb9a2b80a3dc16c677"
format_stamp: "Formatted at 2026-08-09 04:56:06 on dist-test-slave-21b9"
server_key: "49b7c2806a1ca72f9267569b4a1e594d"
server_key_iv: "c91bbe8fdb04cc9ed72b960983a8357f"
server_key_version: "encryptionkey@0"
I20260809 04:56:06.793286 1432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "d40828a794a945fb9a2b80a3dc16c677"
format_stamp: "Formatted at 2026-08-09 04:56:06 on dist-test-slave-21b9"
server_key: "49b7c2806a1ca72f9267569b4a1e594d"
server_key_iv: "c91bbe8fdb04cc9ed72b960983a8357f"
server_key_version: "encryptionkey@0"
I20260809 04:56:06.798548 1432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:56:06.802050 1453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:06.803388 1432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 04:56:06.803558 1432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "d40828a794a945fb9a2b80a3dc16c677"
format_stamp: "Formatted at 2026-08-09 04:56:06 on dist-test-slave-21b9"
server_key: "49b7c2806a1ca72f9267569b4a1e594d"
server_key_iv: "c91bbe8fdb04cc9ed72b960983a8357f"
server_key_version: "encryptionkey@0"
I20260809 04:56:06.803735 1432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:06.822875 1432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:06.827144 1432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:06.827458 1432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:06.830256 1432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:06.830374 1432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:06.830528 1432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:06.830657 1432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 09 04:56:06 dist-test-slave-21b9 krb5kdc[996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:06.851701 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:06.830793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:50003 (local address 127.29.243.254:35799)
0809 04:56:06.831000 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:06.831007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:06.832410 (+ 1403us) server_negotiation.cc:407] Connection header received
0809 04:56:06.834484 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:06.834491 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:06.834596 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:06.834725 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:06.837209 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.838191 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.840226 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.840887 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.841034 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:06.845593 (+ 4559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:06.845600 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:06.845604 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:06.845647 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:06.848694 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:06.849414 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:06.849418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:06.849420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:06.849542 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:06.849965 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:06.849969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:06.849971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:06.850264 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:06.850413 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:06.851099 (+ 686us) server_negotiation.cc:299] Negotiation successful
0809 04:56:06.851356 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:56:06.853111 1469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:06.831246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35799 (local address 127.29.243.194:50003)
0809 04:56:06.832217 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:06.832263 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:06.834164 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:06.835031 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:06.835042 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:06.835847 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:06.836943 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:06.836977 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.838343 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.838348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:06.840058 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:06.840074 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.841189 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.841197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:06.841480 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:06.842308 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:06.842345 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:06.845299 (+ 2954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:06.848836 (+ 3537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:06.848846 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:06.848863 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:06.849285 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:06.849645 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:06.849650 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:06.849654 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:06.849851 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:06.850392 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:06.850399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:06.850748 (+ 349us) client_negotiation.cc:769] Sending connection context
0809 04:56:06.850972 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 04:56:06.851540 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":367,"threads_started":1}
I20260809 04:56:06.880254 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:06.866062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:51965 (local address 127.29.243.254:35799)
0809 04:56:06.866265 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:06.866272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:06.867246 (+ 974us) server_negotiation.cc:407] Connection header received
0809 04:56:06.867467 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:06.867470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:06.867538 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:06.867622 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:06.868856 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.869587 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.871579 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.872012 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.872109 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:06.874229 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:06.874234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:06.874237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:06.874268 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:06.877114 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:06.877928 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:06.877934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:06.877937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:06.878104 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:06.878584 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:06.878590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:06.878594 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:06.879020 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:06.879158 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:06.879700 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 04:56:06.879912 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:56:06.880398 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:06.866051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35799 (local address 127.29.243.194:51965)
0809 04:56:06.867114 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:06.867137 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:06.867336 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:06.867666 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:06.867672 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:06.868310 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:06.868712 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:06.868725 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.869707 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.869713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:06.871424 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:06.871442 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:06.872200 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:06.872208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:06.872406 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:06.873099 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:06.873124 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:06.874045 (+ 921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:06.877243 (+ 3198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:06.877252 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:06.877257 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:06.877805 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:06.878241 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:06.878247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:06.878254 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:06.878448 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:06.879145 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:06.879150 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:06.879376 (+ 226us) client_negotiation.cc:769] Sending connection context
0809 04:56:06.879598 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 04:56:06.879936 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"thread_start_us":697,"threads_started":1}
I20260809 04:56:06.885897 1432 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:44487
I20260809 04:56:06.885949 1657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:44487 every 8 connection(s)
I20260809 04:56:06.887092 1432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:56:06.888979 1658 heartbeater.cc:347] Connected to a master server at 127.29.243.254:35799
I20260809 04:56:06.889230 1658 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:06.889859 1658 heartbeater.cc:511] Master 127.29.243.254:35799 requested a full tablet report, sending...
I20260809 04:56:06.891215 1065 ts_manager.cc:194] Registered new tserver with Master: d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487)
I20260809 04:56:06.892055 1065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:50003
I20260809 04:56:06.893559 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 1432
I20260809 04:56:06.893708 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:56:06.894155 30671 external_mini_cluster.cc:1428] Setting key 639de8aa40368d05b84d7cb160347367
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:56:06.939985 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35799
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:07.269595 1667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:07.269860 1667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:07.269990 1667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:07.277073 1667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:07.277175 1667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:07.277295 1667 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:56:07.277359 1667 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:56:07.277470 1667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:56:07.287261 1667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.243.254:35799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:07.288566 1667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:07.289896 1667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:07.302613 1672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:07.303099 1673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:07.303462 1675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:07.303993 1667 server_base.cc:1034] running on GCE node
I20260809 04:56:07.304564 1667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:07.305465 1667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:07.306664 1667 hybrid_clock.cc:648] HybridClock initialized: now 1786251367306606 us; error 61 us; skew 500 ppm
I20260809 04:56:07.306948 1667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:07 dist-test-slave-21b9 krb5kdc[996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251367, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:07.310081 1667 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:56:07.311587 1667 webserver.cc:460] Webserver started at http://127.29.243.195:38687/ using document root <none> and password file <none>
I20260809 04:56:07.312243 1667 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:07.312361 1667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:07.312667 1667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:07.315568 1667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "ffc03d1ddfac40e99eec11793f4784ef"
format_stamp: "Formatted at 2026-08-09 04:56:07 on dist-test-slave-21b9"
server_key: "a15a6e296f1cce82baf744ea2d0efaeb"
server_key_iv: "528780d0592ba5cfa27f276193b590b8"
server_key_version: "encryptionkey@0"
I20260809 04:56:07.316321 1667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "ffc03d1ddfac40e99eec11793f4784ef"
format_stamp: "Formatted at 2026-08-09 04:56:07 on dist-test-slave-21b9"
server_key: "a15a6e296f1cce82baf744ea2d0efaeb"
server_key_iv: "528780d0592ba5cfa27f276193b590b8"
server_key_version: "encryptionkey@0"
I20260809 04:56:07.321419 1667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:56:07.324889 1684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:07.326154 1667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:56:07.326306 1667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "ffc03d1ddfac40e99eec11793f4784ef"
format_stamp: "Formatted at 2026-08-09 04:56:07 on dist-test-slave-21b9"
server_key: "a15a6e296f1cce82baf744ea2d0efaeb"
server_key_iv: "528780d0592ba5cfa27f276193b590b8"
server_key_version: "encryptionkey@0"
I20260809 04:56:07.326489 1667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:07.340770 1667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:07.344796 1667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:07.345072 1667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:07.347589 1667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:07.347705 1667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:07.347812 1667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:07.347890 1667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:56:07 dist-test-slave-21b9 krb5kdc[996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251367, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:07.370460 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.348280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:52623 (local address 127.29.243.254:35799)
0809 04:56:07.348453 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:07.348459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:07.349869 (+ 1410us) server_negotiation.cc:407] Connection header received
0809 04:56:07.352779 (+ 2910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:07.352786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:07.352885 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:07.353010 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:07.355320 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.356380 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.359226 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.359870 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.360020 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.364189 (+ 4169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:07.364196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:07.364201 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:07.364243 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:07.367156 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:07.368120 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:07.368125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:07.368127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:07.368260 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:07.368657 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:07.368662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:07.368666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:07.368975 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:07.369140 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:07.369833 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 04:56:07.370090 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:56:07.371387 1700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.348804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35799 (local address 127.29.243.195:52623)
0809 04:56:07.349667 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:07.349722 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:07.352368 (+ 2646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:07.353293 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:07.353307 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:07.354132 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:07.355093 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.355123 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.356556 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.356562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.358980 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.359001 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.360128 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.360138 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.360458 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.361233 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:07.361266 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:07.363973 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:07.367310 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:07.367324 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:07.367349 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:07.367965 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:07.368367 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:07.368373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:07.368376 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:07.368542 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:07.369116 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:07.369123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:07.369441 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 04:56:07.369661 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 04:56:07.370142 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":336,"threads_started":1}
I20260809 04:56:07.372444 1426 heartbeater.cc:503] Master 127.29.243.254:35799 was elected leader, sending a full tablet report...
I20260809 04:56:07.391713 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.378888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:58227 (local address 127.29.243.254:35799)
0809 04:56:07.379061 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:07.379066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:07.379910 (+ 844us) server_negotiation.cc:407] Connection header received
0809 04:56:07.380069 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:07.380074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:07.380131 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:07.380208 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:07.381069 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.381737 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.383007 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.383472 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.383599 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.385601 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:07.385607 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:07.385612 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:07.385642 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:07.388972 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:07.389688 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:07.389694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:07.389698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:07.389820 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:07.390232 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:07.390237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:07.390240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:07.390525 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:07.390682 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:07.391198 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 04:56:07.391445 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:56:07.392024 1815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.378656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35799 (local address 127.29.243.195:58227)
0809 04:56:07.379813 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:07.379832 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:07.379953 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:07.380243 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:07.380248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:07.380667 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:07.380938 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.380946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.381867 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.381873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.382880 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.382891 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.383677 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.383683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.383908 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.384605 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:07.384622 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:07.385434 (+ 812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:07.389110 (+ 3676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:07.389116 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:07.389120 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:07.389581 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:07.389955 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:07.389960 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:07.389963 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:07.390126 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:07.390657 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:07.390663 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:07.390900 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 04:56:07.391068 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 04:56:07.391540 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1021,"thread_start_us":854,"threads_started":1}
I20260809 04:56:07.401274 1667 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:41113
I20260809 04:56:07.401324 1891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:41113 every 8 connection(s)
I20260809 04:56:07.402442 1667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:56:07.404412 1892 heartbeater.cc:347] Connected to a master server at 127.29.243.254:35799
I20260809 04:56:07.404651 1892 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:07.405277 1892 heartbeater.cc:511] Master 127.29.243.254:35799 requested a full tablet report, sending...
I20260809 04:56:07.406531 1065 ts_manager.cc:194] Registered new tserver with Master: ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113)
I20260809 04:56:07.407359 1065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:52623
I20260809 04:56:07.409819 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 1667
I20260809 04:56:07.409989 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:56:07.410408 30671 external_mini_cluster.cc:1428] Setting key 8b7044034536e4a890dd6ec00724d0c1
I20260809 04:56:07.413216 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:56:07 dist-test-slave-21b9 krb5kdc[996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:56:07 dist-test-slave-21b9 krb5kdc[996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:07.464900 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.451613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.29.243.254:35799)
0809 04:56:07.451786 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:07.451791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:07.452128 (+ 337us) server_negotiation.cc:407] Connection header received
0809 04:56:07.452311 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:07.452316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:07.452384 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:07.452466 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:07.453852 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.454608 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.455899 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.456331 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.456476 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.460177 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:07.460182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:07.460185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:07.460219 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:07.462725 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:07.463341 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:07.463345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:07.463347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:07.463477 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:07.463765 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:07.463768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:07.463770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:07.464038 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:07.464222 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:07.464437 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 04:56:07.464607 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:56:07.468844 1065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:07.476121 1065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:07.516216 1925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.489561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43692 (local address 127.29.243.194:44487)
0809 04:56:07.506081 (+ 16520us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:07.506091 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:07.507424 (+ 1333us) server_negotiation.cc:407] Connection header received
0809 04:56:07.507541 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:07.507549 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:07.507769 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:07.507966 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:07.509662 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.511058 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.513585 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.514695 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.514871 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.515050 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:07.515753 (+ 703us) server_negotiation.cc:299] Negotiation successful
0809 04:56:07.515930 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16363,"thread_start_us":16255,"threads_started":1}
I20260809 04:56:07.516307 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.488707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:44487 (local address 127.0.0.1:43692)
0809 04:56:07.507075 (+ 18368us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:07.507089 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:07.507211 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:07.507985 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:07.507993 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:07.508023 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:07.509390 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.509431 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.511240 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.511246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.513434 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.513446 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.515260 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.515293 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.515487 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.515537 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 04:56:07.515685 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 04:56:07.515870 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18239,"mutex_wait_us":33,"thread_start_us":19714,"threads_started":1}
I20260809 04:56:07.516654 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.488945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41113 (local address 127.0.0.1:43312)
0809 04:56:07.508740 (+ 19795us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:07.508757 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:07.508881 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:07.509526 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:07.509534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:07.509562 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:07.510000 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.510013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.511909 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.511915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.514017 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.514028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.515759 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.515766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.515940 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.515972 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:56:07.516060 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:56:07.516174 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19678,"thread_start_us":17867,"threads_started":1}
I20260809 04:56:07.516901 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.490012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.29.243.195:41113)
0809 04:56:07.506604 (+ 16592us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:07.506612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:07.508958 (+ 2346us) server_negotiation.cc:407] Connection header received
0809 04:56:07.509089 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:07.509097 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:07.509320 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:07.509535 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:07.510212 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.511769 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.514166 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.515221 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.515426 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.515613 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:07.516404 (+ 791us) server_negotiation.cc:299] Negotiation successful
0809 04:56:07.516572 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16427,"thread_start_us":16254,"threads_started":1}
I20260809 04:56:07.517406 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.488476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45813 (local address 127.0.0.1:58202)
0809 04:56:07.504552 (+ 16076us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:07.504604 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:07.504842 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:07.509220 (+ 4378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:07.509225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:07.509248 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:07.509700 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.509716 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.511362 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.511368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.513952 (+ 2584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.513968 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.516711 (+ 2743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.516716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.516891 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.516922 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 04:56:07.517013 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 04:56:07.517127 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15764,"thread_start_us":16565,"threads_started":1}
I20260809 04:56:07.518118 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.488607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58202 (local address 127.29.243.193:45813)
0809 04:56:07.508659 (+ 20052us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:07.508667 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:07.508696 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:56:07.508797 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:07.508804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:07.509030 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:07.509207 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:07.509924 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.510955 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.514138 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.516169 (+ 2031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.517577 (+ 1408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.517735 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:07.517787 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 04:56:07.517899 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19880,"thread_start_us":19738,"threads_started":1}
I20260809 04:56:07.520723 1538 tablet_service.cc:1467] Processing CreateTablet for tablet 5d201c16e25b436bb452cebe755dd1c6 (DEFAULT_TABLE table=test-table [id=438550f2abb64f2bb3dd7faaa1e54e4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:07.521070 1314 tablet_service.cc:1467] Processing CreateTablet for tablet 5d201c16e25b436bb452cebe755dd1c6 (DEFAULT_TABLE table=test-table [id=438550f2abb64f2bb3dd7faaa1e54e4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:07.522287 1538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d201c16e25b436bb452cebe755dd1c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:07.521929 1774 tablet_service.cc:1467] Processing CreateTablet for tablet 5d201c16e25b436bb452cebe755dd1c6 (DEFAULT_TABLE table=test-table [id=438550f2abb64f2bb3dd7faaa1e54e4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:07.522631 1314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d201c16e25b436bb452cebe755dd1c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:07.523249 1774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d201c16e25b436bb452cebe755dd1c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:07.534137 1927 tablet_bootstrap.cc:492] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: Bootstrap starting.
I20260809 04:56:07.537851 1927 tablet_bootstrap.cc:654] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:07.539237 1927 log.cc:824] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:07.540647 1929 tablet_bootstrap.cc:492] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: Bootstrap starting.
I20260809 04:56:07.542255 1927 tablet_bootstrap.cc:492] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: No bootstrap required, opened a new log
I20260809 04:56:07.542505 1927 ts_tablet_manager.cc:1397] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 04:56:07.544695 1928 tablet_bootstrap.cc:492] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: Bootstrap starting.
I20260809 04:56:07.547386 1928 tablet_bootstrap.cc:654] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:07.548564 1928 log.cc:824] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:07.548949 1929 tablet_bootstrap.cc:654] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:07.550203 1929 log.cc:824] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:07.551232 1927 raft_consensus.cc:348] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.551728 1927 raft_consensus.cc:374] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:07.551851 1927 raft_consensus.cc:729] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a34e4d437d429a84641f1134069a0a, State: Initialized, Role: FOLLOWER
I20260809 04:56:07.552541 1927 consensus_queue.cc:260] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.553257 1929 tablet_bootstrap.cc:492] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: No bootstrap required, opened a new log
I20260809 04:56:07.553551 1928 tablet_bootstrap.cc:492] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: No bootstrap required, opened a new log
I20260809 04:56:07.553594 1929 ts_tablet_manager.cc:1397] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260809 04:56:07.553869 1928 ts_tablet_manager.cc:1397] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260809 04:56:07.554399 1927 ts_tablet_manager.cc:1428] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260809 04:56:07.561757 1929 raft_consensus.cc:348] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.562088 1929 raft_consensus.cc:374] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:07.562186 1929 raft_consensus.cc:729] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc03d1ddfac40e99eec11793f4784ef, State: Initialized, Role: FOLLOWER
I20260809 04:56:07.562486 1928 raft_consensus.cc:348] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.562695 1929 consensus_queue.cc:260] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.562846 1928 raft_consensus.cc:374] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:07.562934 1928 raft_consensus.cc:729] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d40828a794a945fb9a2b80a3dc16c677, State: Initialized, Role: FOLLOWER
I20260809 04:56:07.563472 1928 consensus_queue.cc:260] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.563802 1892 heartbeater.cc:503] Master 127.29.243.254:35799 was elected leader, sending a full tablet report...
I20260809 04:56:07.564297 1929 ts_tablet_manager.cc:1428] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: Time spent starting tablet: real 0.011s user 0.004s sys 0.006s
I20260809 04:56:07.565024 1658 heartbeater.cc:503] Master 127.29.243.254:35799 was elected leader, sending a full tablet report...
I20260809 04:56:07.565596 1928 ts_tablet_manager.cc:1428] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
W20260809 04:56:07.615662 1427 tablet.cc:2369] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:56:07.639062 1661 tablet.cc:2369] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:56:07.653999 1893 tablet.cc:2369] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:56:07.694198 1940 raft_consensus.cc:482] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:07.694558 1940 raft_consensus.cc:504] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.696542 1940 leader_election.cc:288] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487), ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113)
I20260809 04:56:07.703821 1925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.697036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:51935 (local address 127.29.243.194:44487)
0809 04:56:07.697253 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:07.697259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:07.698632 (+ 1373us) server_negotiation.cc:407] Connection header received
0809 04:56:07.698809 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:07.698813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:07.698881 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:07.698953 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:07.699576 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.700325 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.702130 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.702887 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.703031 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.703106 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:07.703442 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 04:56:07.703560 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:56:07.703944 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.697045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:44487 (local address 127.29.243.193:51935)
0809 04:56:07.698519 (+ 1474us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:07.698542 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:07.698680 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:07.698994 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:07.698998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:07.699024 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:07.699419 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.699430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.700463 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.700468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.701975 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.701988 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.703081 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.703087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.703228 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.703312 (+ 84us) client_negotiation.cc:769] Sending connection context
0809 04:56:07.703400 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:56:07.703526 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1353,"mutex_wait_us":75,"thread_start_us":388,"threads_started":1}
I20260809 04:56:07.704041 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.697408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:44225 (local address 127.29.243.195:41113)
0809 04:56:07.697700 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:07.697707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:07.697998 (+ 291us) server_negotiation.cc:407] Connection header received
0809 04:56:07.698384 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:07.698391 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:07.698479 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:07.698556 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:07.699395 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.700203 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.702183 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.702977 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.703143 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.703227 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:07.703593 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 04:56:07.703722 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260809 04:56:07.704604 1946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.696975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41113 (local address 127.29.243.193:44225)
0809 04:56:07.697873 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:07.697898 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:07.698159 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:07.698803 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:07.698807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:07.698841 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:07.699208 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.699226 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.700349 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.700356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.702014 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.702029 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.703213 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.703220 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.703410 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.703442 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:56:07.703534 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:56:07.703665 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":6,"thread_start_us":1169,"threads_started":1}
I20260809 04:56:07.705046 1573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d201c16e25b436bb452cebe755dd1c6" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40828a794a945fb9a2b80a3dc16c677" is_pre_election: true
I20260809 04:56:07.705614 1573 raft_consensus.cc:2454] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a34e4d437d429a84641f1134069a0a in term 0.
I20260809 04:56:07.705579 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d201c16e25b436bb452cebe755dd1c6" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc03d1ddfac40e99eec11793f4784ef" is_pre_election: true
I20260809 04:56:07.706068 1813 raft_consensus.cc:2454] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a34e4d437d429a84641f1134069a0a in term 0.
I20260809 04:56:07.706298 1232 leader_election.cc:302] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a34e4d437d429a84641f1134069a0a, d40828a794a945fb9a2b80a3dc16c677; no voters:
I20260809 04:56:07.706710 1940 raft_consensus.cc:2781] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:07.706852 1940 raft_consensus.cc:482] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:07.706964 1940 raft_consensus.cc:3037] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:07.709331 1940 raft_consensus.cc:504] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.709924 1940 leader_election.cc:288] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 election: Requested vote from peers d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487), ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113)
I20260809 04:56:07.710425 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d201c16e25b436bb452cebe755dd1c6" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc03d1ddfac40e99eec11793f4784ef"
I20260809 04:56:07.710425 1573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d201c16e25b436bb452cebe755dd1c6" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40828a794a945fb9a2b80a3dc16c677"
I20260809 04:56:07.710632 1813 raft_consensus.cc:3037] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:07.710632 1573 raft_consensus.cc:3037] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:07.713793 1573 raft_consensus.cc:2454] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a34e4d437d429a84641f1134069a0a in term 1.
I20260809 04:56:07.713793 1813 raft_consensus.cc:2454] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a34e4d437d429a84641f1134069a0a in term 1.
I20260809 04:56:07.714260 1226 leader_election.cc:302] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 34a34e4d437d429a84641f1134069a0a, d40828a794a945fb9a2b80a3dc16c677, ffc03d1ddfac40e99eec11793f4784ef; no voters:
I20260809 04:56:07.714593 1940 raft_consensus.cc:2781] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:07.715000 1940 raft_consensus.cc:686] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 1 LEADER]: Becoming Leader. State: Replica: 34a34e4d437d429a84641f1134069a0a, State: Running, Role: LEADER
I20260809 04:56:07.715487 1940 consensus_queue.cc:237] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.719995 1064 catalog_manager.cc:5495] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a34e4d437d429a84641f1134069a0a (127.29.243.193). New cstate: current_term: 1 leader_uuid: "34a34e4d437d429a84641f1134069a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } health_report { overall_health: UNKNOWN } } }
Aug 09 04:56:07 dist-test-slave-21b9 krb5kdc[996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251365, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:07.793784 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.774445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55782 (local address 127.29.243.254:35799)
0809 04:56:07.774640 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:07.774650 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:07.775978 (+ 1328us) server_negotiation.cc:407] Connection header received
0809 04:56:07.776158 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:07.776163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:07.776221 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:07.776303 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:07.778584 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.779360 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.781221 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.781648 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.781756 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.786634 (+ 4878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:07.786640 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:07.786643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:07.786674 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:07.790012 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:07.790926 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:07.790937 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:07.790939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:07.791226 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:07.791740 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:07.791745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:07.791747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:07.792140 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:07.792270 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:07.792891 (+ 621us) server_negotiation.cc:299] Negotiation successful
0809 04:56:07.793189 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":75}
I20260809 04:56:07.793859 1964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.774468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35799 (local address 127.0.0.1:55782)
0809 04:56:07.775862 (+ 1394us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:07.775880 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:07.776041 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:07.776344 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:07.776349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:07.777633 (+ 1284us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:07.778426 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.778439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.779477 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.779483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.781081 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.781094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.781950 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.781958 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.782194 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.783009 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:07.783027 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:07.786363 (+ 3336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:07.790182 (+ 3819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:07.790194 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:07.790200 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:07.790798 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:07.791353 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:07.791358 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:07.791363 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:07.791637 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:07.792321 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:07.792329 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:07.792542 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 04:56:07.792744 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 04:56:07.793021 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1167,"thread_start_us":870,"threads_started":1}
I20260809 04:56:07.800644 1064 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:55782:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:56:07.802067 1064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:07.809502 1314 tablet_service.cc:1467] Processing CreateTablet for tablet 720a9123ad6840bf833527e0bde03a34 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=31691b1859fd46b7b1e4eb3f7bd80045]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:56:07.809667 1538 tablet_service.cc:1467] Processing CreateTablet for tablet 720a9123ad6840bf833527e0bde03a34 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=31691b1859fd46b7b1e4eb3f7bd80045]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:56:07.809952 1774 tablet_service.cc:1467] Processing CreateTablet for tablet 720a9123ad6840bf833527e0bde03a34 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=31691b1859fd46b7b1e4eb3f7bd80045]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:56:07.810766 1314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 720a9123ad6840bf833527e0bde03a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:07.810822 1538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 720a9123ad6840bf833527e0bde03a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:07.811103 1774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 720a9123ad6840bf833527e0bde03a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:07.818881 1927 tablet_bootstrap.cc:492] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: Bootstrap starting.
I20260809 04:56:07.821163 1929 tablet_bootstrap.cc:492] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef: Bootstrap starting.
I20260809 04:56:07.821565 1928 tablet_bootstrap.cc:492] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677: Bootstrap starting.
I20260809 04:56:07.821933 1927 tablet_bootstrap.cc:654] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:07.824700 1928 tablet_bootstrap.cc:654] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:07.824721 1929 tablet_bootstrap.cc:654] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:07.829252 1928 tablet_bootstrap.cc:492] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677: No bootstrap required, opened a new log
I20260809 04:56:07.829363 1929 tablet_bootstrap.cc:492] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef: No bootstrap required, opened a new log
I20260809 04:56:07.829492 1928 ts_tablet_manager.cc:1397] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260809 04:56:07.829567 1929 ts_tablet_manager.cc:1397] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 04:56:07.829658 1927 tablet_bootstrap.cc:492] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: No bootstrap required, opened a new log
I20260809 04:56:07.829887 1927 ts_tablet_manager.cc:1397] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260809 04:56:07.830905 1928 raft_consensus.cc:348] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.830979 1929 raft_consensus.cc:348] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.831307 1928 raft_consensus.cc:374] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:07.831393 1929 raft_consensus.cc:374] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:07.831305 1927 raft_consensus.cc:348] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.831539 1928 raft_consensus.cc:729] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d40828a794a945fb9a2b80a3dc16c677, State: Initialized, Role: FOLLOWER
I20260809 04:56:07.831624 1929 raft_consensus.cc:729] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc03d1ddfac40e99eec11793f4784ef, State: Initialized, Role: FOLLOWER
I20260809 04:56:07.831697 1927 raft_consensus.cc:374] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:07.831902 1927 raft_consensus.cc:729] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a34e4d437d429a84641f1134069a0a, State: Initialized, Role: FOLLOWER
I20260809 04:56:07.831970 1929 consensus_queue.cc:260] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.832048 1928 consensus_queue.cc:260] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.833012 1929 ts_tablet_manager.cc:1428] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:07.832688 1943 tablet_replica.cc:439] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } } }
I20260809 04:56:07.832705 1944 tablet_replica.cc:439] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } } }
I20260809 04:56:07.833532 1943 tablet_replica.cc:442] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:07.833652 1944 tablet_replica.cc:442] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:07.833314 1927 consensus_queue.cc:260] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.834446 1940 tablet_replica.cc:439] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } } }
I20260809 04:56:07.835130 1940 tablet_replica.cc:442] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:07.834663 1927 ts_tablet_manager.cc:1428] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:56:07.836071 1928 ts_tablet_manager.cc:1428] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260809 04:56:07.876444 1940 raft_consensus.cc:482] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:07.876698 1940 raft_consensus.cc:504] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.877473 1940 leader_election.cc:288] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487), ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113)
I20260809 04:56:07.877986 1573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720a9123ad6840bf833527e0bde03a34" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40828a794a945fb9a2b80a3dc16c677" is_pre_election: true
I20260809 04:56:07.878042 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720a9123ad6840bf833527e0bde03a34" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc03d1ddfac40e99eec11793f4784ef" is_pre_election: true
I20260809 04:56:07.878367 1573 raft_consensus.cc:2454] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a34e4d437d429a84641f1134069a0a in term 0.
I20260809 04:56:07.878393 1813 raft_consensus.cc:2454] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a34e4d437d429a84641f1134069a0a in term 0.
I20260809 04:56:07.878813 1232 leader_election.cc:302] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a34e4d437d429a84641f1134069a0a, d40828a794a945fb9a2b80a3dc16c677; no voters:
I20260809 04:56:07.879192 1940 raft_consensus.cc:2781] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:07.879369 1940 raft_consensus.cc:482] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:07.879606 1940 raft_consensus.cc:3037] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:07.882762 1940 raft_consensus.cc:504] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.883466 1940 leader_election.cc:288] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 election: Requested vote from peers d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487), ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113)
I20260809 04:56:07.883958 1573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720a9123ad6840bf833527e0bde03a34" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40828a794a945fb9a2b80a3dc16c677"
I20260809 04:56:07.884016 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720a9123ad6840bf833527e0bde03a34" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc03d1ddfac40e99eec11793f4784ef"
I20260809 04:56:07.884297 1573 raft_consensus.cc:3037] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:07.884338 1813 raft_consensus.cc:3037] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:07.887678 1573 raft_consensus.cc:2454] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a34e4d437d429a84641f1134069a0a in term 1.
I20260809 04:56:07.888188 1232 leader_election.cc:302] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a34e4d437d429a84641f1134069a0a, d40828a794a945fb9a2b80a3dc16c677; no voters:
I20260809 04:56:07.888214 1813 raft_consensus.cc:2454] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a34e4d437d429a84641f1134069a0a in term 1.
I20260809 04:56:07.888549 1940 raft_consensus.cc:2781] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:07.888960 1940 raft_consensus.cc:686] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [term 1 LEADER]: Becoming Leader. State: Replica: 34a34e4d437d429a84641f1134069a0a, State: Running, Role: LEADER
I20260809 04:56:07.889470 1940 consensus_queue.cc:237] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:07.892658 1949 tablet_replica.cc:439] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: TxnStatusTablet state changed. Reason: New leader 34a34e4d437d429a84641f1134069a0a. Latest consensus state: current_term: 1 leader_uuid: "34a34e4d437d429a84641f1134069a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } } }
I20260809 04:56:07.893115 1949 tablet_replica.cc:442] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:56:07.893971 1063 catalog_manager.cc:5495] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a34e4d437d429a84641f1134069a0a (127.29.243.193). New cstate: current_term: 1 leader_uuid: "34a34e4d437d429a84641f1134069a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } health_report { overall_health: UNKNOWN } } }
I20260809 04:56:07.894784 1974 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:56:07.895205 1974 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:56:07.928941 1964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.920347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45813 (local address 127.0.0.1:58214)
0809 04:56:07.920661 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:07.920679 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:07.920793 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:07.921369 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:07.921373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:07.921385 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:56:07.921712 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.921723 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.923567 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.923571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.925005 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:07.925019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.926463 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.926469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:07.926787 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.926881 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:56:07.928256 (+ 1375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:56:07.928275 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:56:07.928334 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 04:56:07.928524 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 04:56:07.928958 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.920469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58214 (local address 127.29.243.193:45813)
0809 04:56:07.920842 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:07.920850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:07.920875 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:56:07.921007 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:07.921014 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:07.921143 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:07.921350 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:07.921874 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.923435 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.925163 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.926229 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.926420 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.927316 (+ 896us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:07.928142 (+ 826us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:07.928351 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:07.928404 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 04:56:07.928505 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:56:07.940306 1573 raft_consensus.cc:1261] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Refusing update from remote peer 34a34e4d437d429a84641f1134069a0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:07.940399 1813 raft_consensus.cc:1261] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Refusing update from remote peer 34a34e4d437d429a84641f1134069a0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:07.941689 1940 consensus_queue.cc:1035] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:07.942224 1949 consensus_queue.cc:1035] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:07.954080 1944 tablet_replica.cc:439] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677: TxnStatusTablet state changed. Reason: New leader 34a34e4d437d429a84641f1134069a0a. Latest consensus state: current_term: 1 leader_uuid: "34a34e4d437d429a84641f1134069a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } } }
I20260809 04:56:07.954569 1944 tablet_replica.cc:442] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:07.959676 1944 tablet_replica.cc:439] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34a34e4d437d429a84641f1134069a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } } }
I20260809 04:56:07.960155 1944 tablet_replica.cc:442] T 720a9123ad6840bf833527e0bde03a34 P d40828a794a945fb9a2b80a3dc16c677: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:07.960660 1978 mvcc.cc:205] Tried to move back new op lower bound from 7316485603069882368 to 7316485602878869504. Current Snapshot: MvccSnapshot[applied={T|T < 7316485603069882368}]
I20260809 04:56:07.963567 1943 tablet_replica.cc:439] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef: TxnStatusTablet state changed. Reason: New leader 34a34e4d437d429a84641f1134069a0a. Latest consensus state: current_term: 1 leader_uuid: "34a34e4d437d429a84641f1134069a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } } }
I20260809 04:56:07.963896 1943 tablet_replica.cc:442] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:07.972342 1949 tablet_replica.cc:439] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34a34e4d437d429a84641f1134069a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } } }
I20260809 04:56:07.972784 1949 tablet_replica.cc:442] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:56:07.974452 1949 tablet_replica.cc:439] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34a34e4d437d429a84641f1134069a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } } }
I20260809 04:56:07.975201 1949 tablet_replica.cc:442] T 720a9123ad6840bf833527e0bde03a34 P 34a34e4d437d429a84641f1134069a0a: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:56:07.980140 1943 tablet_replica.cc:439] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34a34e4d437d429a84641f1134069a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } } }
I20260809 04:56:07.980536 1943 tablet_replica.cc:442] T 720a9123ad6840bf833527e0bde03a34 P ffc03d1ddfac40e99eec11793f4784ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:07.982041 1981 mvcc.cc:205] Tried to move back new op lower bound from 7316485603069882368 to 7316485602878869504. Current Snapshot: MvccSnapshot[applied={T|T < 7316485603069882368}]
I20260809 04:56:07.997478 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:07.986262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58216 (local address 127.29.243.193:45813)
0809 04:56:07.986485 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:07.986492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:07.987360 (+ 868us) server_negotiation.cc:407] Connection header received
0809 04:56:07.987780 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:07.987787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:07.987885 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:07.987957 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:07.989147 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.991202 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.994482 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:07.995539 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:07.995715 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:07.996256 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:07.996415 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:07.996521 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:07.996828 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 04:56:07.996929 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 04:56:08.027587 1997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:08.019155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45813 (local address 127.29.243.193:41143)
0809 04:56:08.020238 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:08.020260 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:08.020462 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:08.020893 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:08.020899 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:08.020914 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:56:08.021455 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:08.021472 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.022994 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.023000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:08.025033 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:08.025051 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.026117 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.026124 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:08.026314 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:08.026406 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:56:08.026966 (+ 560us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:56:08.026975 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:56:08.027045 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 04:56:08.027199 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":633,"threads_started":1}
I20260809 04:56:08.028122 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:08.018808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:41143 (local address 127.29.243.193:45813)
0809 04:56:08.019010 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:08.019017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:08.020372 (+ 1355us) server_negotiation.cc:407] Connection header received
0809 04:56:08.020619 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:08.020626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:08.020732 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:08.020858 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:08.021663 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.022847 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.025183 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.025880 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.026020 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:08.026634 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:08.026821 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:08.026928 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:08.027812 (+ 884us) server_negotiation.cc:299] Negotiation successful
0809 04:56:08.027899 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:56:08.043983 1812 raft_consensus.cc:1261] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Refusing update from remote peer 34a34e4d437d429a84641f1134069a0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:08.043974 1573 raft_consensus.cc:1261] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Refusing update from remote peer 34a34e4d437d429a84641f1134069a0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:08.044696 1940 consensus_queue.cc:1035] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:08.045186 1988 consensus_queue.cc:1035] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:08.117764 1065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:08.118214 1065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:08.122783 1065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5: Sending DeleteTablet for 3 replicas of tablet 5d201c16e25b436bb452cebe755dd1c6
I20260809 04:56:08.124140 1313 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d201c16e25b436bb452cebe755dd1c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:08 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:58202
I20260809 04:56:08.124673 1538 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d201c16e25b436bb452cebe755dd1c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:08 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:43692
I20260809 04:56:08.125259 1774 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d201c16e25b436bb452cebe755dd1c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:08 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:43312
I20260809 04:56:08.126539 1065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:08.127629 1065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:08.125680 2010 tablet_replica.cc:330] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: stopping tablet replica
I20260809 04:56:08.132877 2012 tablet_replica.cc:330] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: stopping tablet replica
I20260809 04:56:08.133416 2012 raft_consensus.cc:2229] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:08.133507 2010 raft_consensus.cc:2229] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:08.133890 2010 raft_consensus.cc:2258] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:08.133893 2012 raft_consensus.cc:2258] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:08.136181 1538 tablet_service.cc:1467] Processing CreateTablet for tablet 72436fa13d2940c28b9bab7c035be9b9 (DEFAULT_TABLE table=test-table [id=7bccc44312434031b82360271352ee62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:08.136701 1538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72436fa13d2940c28b9bab7c035be9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:08.136633 1772 tablet_service.cc:1467] Processing CreateTablet for tablet 72436fa13d2940c28b9bab7c035be9b9 (DEFAULT_TABLE table=test-table [id=7bccc44312434031b82360271352ee62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:08.137132 1772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72436fa13d2940c28b9bab7c035be9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:08.141194 1314 tablet_service.cc:1467] Processing CreateTablet for tablet 72436fa13d2940c28b9bab7c035be9b9 (DEFAULT_TABLE table=test-table [id=7bccc44312434031b82360271352ee62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:08.141678 1314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72436fa13d2940c28b9bab7c035be9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:08.141678 2010 ts_tablet_manager.cc:1905] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:08.144205 1929 tablet_bootstrap.cc:492] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef: Bootstrap starting.
I20260809 04:56:08.145046 2011 tablet_replica.cc:330] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: stopping tablet replica
I20260809 04:56:08.145501 2011 raft_consensus.cc:2229] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:08.145774 1929 tablet_bootstrap.cc:654] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:08.146243 2011 raft_consensus.cc:2258] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:08.148682 2011 ts_tablet_manager.cc:1905] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:08.148808 1929 tablet_bootstrap.cc:492] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef: No bootstrap required, opened a new log
I20260809 04:56:08.149047 1929 ts_tablet_manager.cc:1397] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:56:08.149969 1929 raft_consensus.cc:348] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:08.150252 1929 raft_consensus.cc:374] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:08.150281 2012 ts_tablet_manager.cc:1905] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:08.150365 1929 raft_consensus.cc:729] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc03d1ddfac40e99eec11793f4784ef, State: Initialized, Role: FOLLOWER
I20260809 04:56:08.150946 1928 tablet_bootstrap.cc:492] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677: Bootstrap starting.
I20260809 04:56:08.152101 1929 consensus_queue.cc:260] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:08.153079 1928 tablet_bootstrap.cc:654] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:08.155488 2010 ts_tablet_manager.cc:1918] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:08.155679 2010 log.cc:1196] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/5d201c16e25b436bb452cebe755dd1c6
I20260809 04:56:08.156219 2010 ts_tablet_manager.cc:1939] T 5d201c16e25b436bb452cebe755dd1c6 P d40828a794a945fb9a2b80a3dc16c677: Deleting consensus metadata
I20260809 04:56:08.157388 1929 ts_tablet_manager.cc:1428] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260809 04:56:08.157547 1927 tablet_bootstrap.cc:492] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a: Bootstrap starting.
I20260809 04:56:08.159296 1927 tablet_bootstrap.cc:654] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:08.161978 1046 catalog_manager.cc:4841] TS d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487): tablet 5d201c16e25b436bb452cebe755dd1c6 (table test-table [id=438550f2abb64f2bb3dd7faaa1e54e4d]) successfully deleted
I20260809 04:56:08.162401 2011 ts_tablet_manager.cc:1918] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:08.162667 2011 log.cc:1196] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/5d201c16e25b436bb452cebe755dd1c6
I20260809 04:56:08.162984 1928 tablet_bootstrap.cc:492] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677: No bootstrap required, opened a new log
I20260809 04:56:08.163169 1928 ts_tablet_manager.cc:1397] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.005s
I20260809 04:56:08.163400 2011 ts_tablet_manager.cc:1939] T 5d201c16e25b436bb452cebe755dd1c6 P 34a34e4d437d429a84641f1134069a0a: Deleting consensus metadata
I20260809 04:56:08.163646 1927 tablet_bootstrap.cc:492] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a: No bootstrap required, opened a new log
I20260809 04:56:08.163812 1927 ts_tablet_manager.cc:1397] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:56:08.164027 1928 raft_consensus.cc:348] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:08.164309 1928 raft_consensus.cc:374] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:08.164397 1928 raft_consensus.cc:729] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d40828a794a945fb9a2b80a3dc16c677, State: Initialized, Role: FOLLOWER
I20260809 04:56:08.164552 1927 raft_consensus.cc:348] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:08.164803 1927 raft_consensus.cc:374] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:08.164711 1928 consensus_queue.cc:260] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:08.164886 1927 raft_consensus.cc:729] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a34e4d437d429a84641f1134069a0a, State: Initialized, Role: FOLLOWER
I20260809 04:56:08.165174 1927 consensus_queue.cc:260] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:08.165481 1928 ts_tablet_manager.cc:1428] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 04:56:08.165956 2012 ts_tablet_manager.cc:1918] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:08.166148 2012 log.cc:1196] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/5d201c16e25b436bb452cebe755dd1c6
I20260809 04:56:08.166531 1927 ts_tablet_manager.cc:1428] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:08.166795 2012 ts_tablet_manager.cc:1939] T 5d201c16e25b436bb452cebe755dd1c6 P ffc03d1ddfac40e99eec11793f4784ef: Deleting consensus metadata
I20260809 04:56:08.168324 1043 catalog_manager.cc:4841] TS 34a34e4d437d429a84641f1134069a0a (127.29.243.193:45813): tablet 5d201c16e25b436bb452cebe755dd1c6 (table test-table [id=438550f2abb64f2bb3dd7faaa1e54e4d]) successfully deleted
I20260809 04:56:08.168673 1045 catalog_manager.cc:4841] TS ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113): tablet 5d201c16e25b436bb452cebe755dd1c6 (table test-table [id=438550f2abb64f2bb3dd7faaa1e54e4d]) successfully deleted
I20260809 04:56:08.280313 1944 raft_consensus.cc:482] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:08.280604 1944 raft_consensus.cc:504] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:08.282545 1944 leader_election.cc:288] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34a34e4d437d429a84641f1134069a0a (127.29.243.193:45813), ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113)
I20260809 04:56:08.290628 2018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:08.282765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41113 (local address 127.29.243.194:55535)
0809 04:56:08.283672 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:08.283692 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:08.283836 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:08.284989 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:08.284994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:08.285022 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:08.285327 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:08.285340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.286502 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.286505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:08.288381 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:08.288397 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.289871 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.289878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:08.290042 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:08.290078 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 04:56:08.290174 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 04:56:08.290304 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":581,"threads_started":1}
I20260809 04:56:08.290769 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:08.283049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:55535 (local address 127.29.243.195:41113)
0809 04:56:08.284620 (+ 1571us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:08.284626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:08.284667 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:56:08.284747 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:08.284752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:08.284846 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:08.284959 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:08.285541 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.286371 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.288557 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.289691 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.289850 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:08.289935 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:08.290264 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 04:56:08.290405 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1468,"thread_start_us":1351,"threads_started":1}
I20260809 04:56:08.291347 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:08.283044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:59999 (local address 127.29.243.193:45813)
0809 04:56:08.283231 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:08.283238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:08.284554 (+ 1316us) server_negotiation.cc:407] Connection header received
0809 04:56:08.284825 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:08.284835 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:08.284916 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:08.284995 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:08.285738 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.286849 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.289000 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.289945 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.290106 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:08.290176 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:08.290603 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 04:56:08.291001 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:56:08.291428 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72436fa13d2940c28b9bab7c035be9b9" candidate_uuid: "d40828a794a945fb9a2b80a3dc16c677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc03d1ddfac40e99eec11793f4784ef" is_pre_election: true
I20260809 04:56:08.291710 1813 raft_consensus.cc:2454] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d40828a794a945fb9a2b80a3dc16c677 in term 0.
I20260809 04:56:08.292303 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:08.282765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45813 (local address 127.29.243.194:59999)
0809 04:56:08.284437 (+ 1672us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:08.284455 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:08.284659 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:08.285145 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:08.285149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:08.285170 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:08.285566 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:08.285578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.287001 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.287006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:08.288863 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:08.288874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.290235 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.290243 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:08.290412 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:08.290444 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:56:08.291374 (+ 930us) client_negotiation.cc:240] Negotiation successful
0809 04:56:08.291498 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1551,"mutex_wait_us":207,"thread_start_us":1132,"threads_started":1}
I20260809 04:56:08.292332 1457 leader_election.cc:302] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d40828a794a945fb9a2b80a3dc16c677, ffc03d1ddfac40e99eec11793f4784ef; no voters:
I20260809 04:56:08.292850 1944 raft_consensus.cc:2781] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:08.292977 1944 raft_consensus.cc:482] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:08.293066 1944 raft_consensus.cc:3037] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:08.293558 1349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72436fa13d2940c28b9bab7c035be9b9" candidate_uuid: "d40828a794a945fb9a2b80a3dc16c677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a34e4d437d429a84641f1134069a0a" is_pre_election: true
I20260809 04:56:08.294013 1349 raft_consensus.cc:2454] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d40828a794a945fb9a2b80a3dc16c677 in term 0.
I20260809 04:56:08.295874 1944 raft_consensus.cc:504] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:08.296519 1944 leader_election.cc:288] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [CANDIDATE]: Term 1 election: Requested vote from peers 34a34e4d437d429a84641f1134069a0a (127.29.243.193:45813), ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113)
I20260809 04:56:08.296955 1349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72436fa13d2940c28b9bab7c035be9b9" candidate_uuid: "d40828a794a945fb9a2b80a3dc16c677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a34e4d437d429a84641f1134069a0a"
I20260809 04:56:08.297075 1813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72436fa13d2940c28b9bab7c035be9b9" candidate_uuid: "d40828a794a945fb9a2b80a3dc16c677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc03d1ddfac40e99eec11793f4784ef"
I20260809 04:56:08.297178 1349 raft_consensus.cc:3037] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:08.297295 1813 raft_consensus.cc:3037] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:08.300631 1349 raft_consensus.cc:2454] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d40828a794a945fb9a2b80a3dc16c677 in term 1.
I20260809 04:56:08.301121 1454 leader_election.cc:302] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a34e4d437d429a84641f1134069a0a, d40828a794a945fb9a2b80a3dc16c677; no voters:
I20260809 04:56:08.301183 1813 raft_consensus.cc:2454] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d40828a794a945fb9a2b80a3dc16c677 in term 1.
I20260809 04:56:08.301484 1944 raft_consensus.cc:2781] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:08.301975 1944 raft_consensus.cc:686] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 1 LEADER]: Becoming Leader. State: Replica: d40828a794a945fb9a2b80a3dc16c677, State: Running, Role: LEADER
I20260809 04:56:08.302346 1944 consensus_queue.cc:237] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:08.306320 1063 catalog_manager.cc:5495] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 reported cstate change: term changed from 0 to 1, leader changed from <none> to d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194). New cstate: current_term: 1 leader_uuid: "d40828a794a945fb9a2b80a3dc16c677" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } health_report { overall_health: UNKNOWN } } }
I20260809 04:56:08.349952 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:08.343836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43708 (local address 127.29.243.194:44487)
0809 04:56:08.344742 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:08.344746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:08.344764 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:56:08.344845 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:08.344848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:08.344928 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:08.345082 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:08.345499 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.346247 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.347621 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.348053 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.348160 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:08.348621 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:08.349294 (+ 673us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:08.349439 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:08.349524 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 04:56:08.349608 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":685,"threads_started":1}
I20260809 04:56:08.369850 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:08.362603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:36113 (local address 127.29.243.193:45813)
0809 04:56:08.362805 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:08.362811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:08.363460 (+ 649us) server_negotiation.cc:407] Connection header received
0809 04:56:08.363709 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:08.363715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:08.363801 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:08.363895 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:08.364498 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.365671 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.367679 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.368351 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.368496 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:08.369006 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:08.369223 (+ 217us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:08.369304 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:08.369479 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 04:56:08.369567 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 04:56:08.370019 2029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:08.361893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45813 (local address 127.29.243.194:36113)
0809 04:56:08.363341 (+ 1448us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:08.363363 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:08.363526 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:08.363941 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:08.363946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:08.363959 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:56:08.364324 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:08.364338 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.365839 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.365845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:08.367498 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:08.367514 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.368544 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.368551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:08.368733 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:08.368811 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:56:08.369334 (+ 523us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:56:08.369355 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:56:08.369450 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 04:56:08.369591 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1221,"thread_start_us":1046,"threads_started":1}
I20260809 04:56:08.380995 1813 raft_consensus.cc:1261] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Refusing update from remote peer d40828a794a945fb9a2b80a3dc16c677: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:08.381860 1349 raft_consensus.cc:1261] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Refusing update from remote peer d40828a794a945fb9a2b80a3dc16c677: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:08.382409 2021 consensus_queue.cc:1035] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:08.383297 2021 consensus_queue.cc:1035] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:08.397325 1979 mvcc.cc:205] Tried to move back new op lower bound from 7316485604882890752 to 7316485604569903104. Current Snapshot: MvccSnapshot[applied={T|T < 7316485604882890752}]
I20260809 04:56:08.427690 1997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:08.419330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:44487 (local address 127.29.243.193:36301)
0809 04:56:08.419632 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:08.419651 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:08.419757 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:08.421844 (+ 2087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:08.421849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:08.421863 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:56:08.422087 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:08.422099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.423491 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.423497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:08.425297 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:08.425315 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.426375 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.426382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:08.426620 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:08.426672 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:56:08.427109 (+ 437us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:56:08.427121 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 04:56:08.427192 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 04:56:08.427360 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260809 04:56:08.428504 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:08.421121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:36301 (local address 127.29.243.194:44487)
0809 04:56:08.421290 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:08.421297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:08.421315 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:56:08.421380 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:08.421386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:08.421464 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:08.421553 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:08.422253 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.423318 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.425488 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:08.426114 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:08.426242 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:08.426850 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:08.426982 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:08.427061 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:08.428159 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0809 04:56:08.428242 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 04:56:08.430481 1979 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 04:56:10.058372 1063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:10.058686 1063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:10.065650 1063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5: Sending DeleteTablet for 3 replicas of tablet 72436fa13d2940c28b9bab7c035be9b9
I20260809 04:56:10.066672 1314 tablet_service.cc:1514] Processing DeleteTablet for tablet 72436fa13d2940c28b9bab7c035be9b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:10 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:58202
I20260809 04:56:10.066831 1538 tablet_service.cc:1514] Processing DeleteTablet for tablet 72436fa13d2940c28b9bab7c035be9b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:10 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:43692
I20260809 04:56:10.066960 1774 tablet_service.cc:1514] Processing DeleteTablet for tablet 72436fa13d2940c28b9bab7c035be9b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:10 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:43312
I20260809 04:56:10.079991 2084 tablet_replica.cc:330] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a: stopping tablet replica
I20260809 04:56:10.080274 2088 tablet_replica.cc:330] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef: stopping tablet replica
I20260809 04:56:10.080516 2084 raft_consensus.cc:2229] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:10.080705 2088 raft_consensus.cc:2229] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:10.081214 2084 raft_consensus.cc:2258] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:10.081298 2088 raft_consensus.cc:2258] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:10.082595 2088 ts_tablet_manager.cc:1905] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:10.082727 2086 tablet_replica.cc:330] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677: stopping tablet replica
I20260809 04:56:10.082939 2084 ts_tablet_manager.cc:1905] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:10.083068 2086 raft_consensus.cc:2229] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:10.084082 2086 raft_consensus.cc:2258] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:10.085160 2086 ts_tablet_manager.cc:1905] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:10.094085 2084 ts_tablet_manager.cc:1918] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:56:10.094310 2084 log.cc:1196] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/72436fa13d2940c28b9bab7c035be9b9
I20260809 04:56:10.094959 2084 ts_tablet_manager.cc:1939] T 72436fa13d2940c28b9bab7c035be9b9 P 34a34e4d437d429a84641f1134069a0a: Deleting consensus metadata
Aug 09 04:56:10 dist-test-slave-21b9 krb5kdc[996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260809 04:56:10.191026 1043 catalog_manager.cc:4841] TS 34a34e4d437d429a84641f1134069a0a (127.29.243.193:45813): tablet 72436fa13d2940c28b9bab7c035be9b9 (table test-table [id=7bccc44312434031b82360271352ee62]) successfully deleted
W20260809 04:56:10.193161 30671 mini_kdc.cc:317] Time spent kinit for test-user: real 0.125s user 0.003s sys 0.014s
I20260809 04:56:10.193302 2088 ts_tablet_manager.cc:1918] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:56:10.193302 2086 ts_tablet_manager.cc:1918] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:56:10.193480 2088 log.cc:1196] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/72436fa13d2940c28b9bab7c035be9b9
I20260809 04:56:10.193480 2086 log.cc:1196] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/72436fa13d2940c28b9bab7c035be9b9
I20260809 04:56:10.193933 2088 ts_tablet_manager.cc:1939] T 72436fa13d2940c28b9bab7c035be9b9 P ffc03d1ddfac40e99eec11793f4784ef: Deleting consensus metadata
I20260809 04:56:10.193933 2086 ts_tablet_manager.cc:1939] T 72436fa13d2940c28b9bab7c035be9b9 P d40828a794a945fb9a2b80a3dc16c677: Deleting consensus metadata
I20260809 04:56:10.195168 1045 catalog_manager.cc:4841] TS ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113): tablet 72436fa13d2940c28b9bab7c035be9b9 (table test-table [id=7bccc44312434031b82360271352ee62]) successfully deleted
I20260809 04:56:10.195168 1046 catalog_manager.cc:4841] TS d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487): tablet 72436fa13d2940c28b9bab7c035be9b9 (table test-table [id=7bccc44312434031b82360271352ee62]) successfully deleted
Aug 09 04:56:10 dist-test-slave-21b9 krb5kdc[996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:10.216603 2102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:10.197566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55794 (local address 127.29.243.254:35799)
0809 04:56:10.198526 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:10.198533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:10.198555 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:56:10.198605 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:10.198609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:10.198685 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:10.198784 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:10.200395 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.201221 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.202870 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.203688 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.203859 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:10.208768 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:10.208773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:10.208777 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:10.208813 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:10.212911 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:10.213738 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:10.213743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:10.213745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:10.214021 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:10.214495 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:10.214499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:10.214502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:10.214927 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:10.215172 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:10.215368 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 04:56:10.215621 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":659,"threads_started":1}
I20260809 04:56:10.220504 1063 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:10.221274 1063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:10.229220 1314 tablet_service.cc:1467] Processing CreateTablet for tablet 2f4a316411c94a8ebf4fda1fc3c59fb4 (DEFAULT_TABLE table=test-table [id=80c70d3960954d9389017a80fbe4b477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:10.229322 1538 tablet_service.cc:1467] Processing CreateTablet for tablet 2f4a316411c94a8ebf4fda1fc3c59fb4 (DEFAULT_TABLE table=test-table [id=80c70d3960954d9389017a80fbe4b477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:10.229382 1774 tablet_service.cc:1467] Processing CreateTablet for tablet 2f4a316411c94a8ebf4fda1fc3c59fb4 (DEFAULT_TABLE table=test-table [id=80c70d3960954d9389017a80fbe4b477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:10.229802 1774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f4a316411c94a8ebf4fda1fc3c59fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:10.229895 1538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f4a316411c94a8ebf4fda1fc3c59fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:10.229946 1314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f4a316411c94a8ebf4fda1fc3c59fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:10.240151 2103 tablet_bootstrap.cc:492] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef: Bootstrap starting.
I20260809 04:56:10.241209 2104 tablet_bootstrap.cc:492] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a: Bootstrap starting.
I20260809 04:56:10.244416 2103 tablet_bootstrap.cc:654] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:10.244719 2104 tablet_bootstrap.cc:654] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:10.246899 2105 tablet_bootstrap.cc:492] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677: Bootstrap starting.
I20260809 04:56:10.248880 2103 tablet_bootstrap.cc:492] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef: No bootstrap required, opened a new log
I20260809 04:56:10.249152 2103 ts_tablet_manager.cc:1397] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260809 04:56:10.249270 2104 tablet_bootstrap.cc:492] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a: No bootstrap required, opened a new log
I20260809 04:56:10.249647 2104 ts_tablet_manager.cc:1397] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260809 04:56:10.251103 2105 tablet_bootstrap.cc:654] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:10.250924 2103 raft_consensus.cc:348] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.251211 2104 raft_consensus.cc:348] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.251581 2104 raft_consensus.cc:374] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:10.251394 2103 raft_consensus.cc:374] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:10.251714 2104 raft_consensus.cc:729] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a34e4d437d429a84641f1134069a0a, State: Initialized, Role: FOLLOWER
I20260809 04:56:10.251899 2103 raft_consensus.cc:729] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc03d1ddfac40e99eec11793f4784ef, State: Initialized, Role: FOLLOWER
I20260809 04:56:10.252089 2104 consensus_queue.cc:260] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.252296 2103 consensus_queue.cc:260] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.253077 2104 ts_tablet_manager.cc:1428] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:10.254745 2103 ts_tablet_manager.cc:1428] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef: Time spent starting tablet: real 0.005s user 0.005s sys 0.002s
I20260809 04:56:10.257731 2105 tablet_bootstrap.cc:492] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677: No bootstrap required, opened a new log
I20260809 04:56:10.258029 2105 ts_tablet_manager.cc:1397] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20260809 04:56:10.259509 2105 raft_consensus.cc:348] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.259881 2105 raft_consensus.cc:374] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:10.260064 2105 raft_consensus.cc:729] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d40828a794a945fb9a2b80a3dc16c677, State: Initialized, Role: FOLLOWER
I20260809 04:56:10.260342 2105 consensus_queue.cc:260] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.261060 2105 ts_tablet_manager.cc:1428] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260809 04:56:10.494001 2109 raft_consensus.cc:482] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:10.494382 2109 raft_consensus.cc:504] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.496264 2109 leader_election.cc:288] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34a34e4d437d429a84641f1134069a0a (127.29.243.193:45813), d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487)
I20260809 04:56:10.515902 2075 raft_consensus.cc:482] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:10.516302 2075 raft_consensus.cc:504] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.517536 2075 leader_election.cc:288] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487), ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113)
I20260809 04:56:10.517976 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4a316411c94a8ebf4fda1fc3c59fb4" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc03d1ddfac40e99eec11793f4784ef" is_pre_election: true
I20260809 04:56:10.518364 1812 raft_consensus.cc:2454] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a34e4d437d429a84641f1134069a0a in term 0.
I20260809 04:56:10.518733 1573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4a316411c94a8ebf4fda1fc3c59fb4" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40828a794a945fb9a2b80a3dc16c677" is_pre_election: true
I20260809 04:56:10.518944 1226 leader_election.cc:302] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a34e4d437d429a84641f1134069a0a, ffc03d1ddfac40e99eec11793f4784ef; no voters:
I20260809 04:56:10.519119 1573 raft_consensus.cc:2454] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a34e4d437d429a84641f1134069a0a in term 0.
I20260809 04:56:10.519232 2113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:10.496624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:52523 (local address 127.29.243.193:45813)
0809 04:56:10.510169 (+ 13545us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:10.510183 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:10.510263 (+ 80us) server_negotiation.cc:407] Connection header received
0809 04:56:10.510341 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:10.510346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:10.510423 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:10.510552 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:10.511296 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.513059 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.515087 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.516917 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.517188 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:10.517269 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:10.518432 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0809 04:56:10.518629 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13277,"thread_start_us":13615,"threads_started":1}
I20260809 04:56:10.519366 2111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:10.496607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:44487 (local address 127.29.243.195:59703)
0809 04:56:10.509162 (+ 12555us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:10.509184 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:10.509339 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:10.510233 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:10.510238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:10.510270 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:10.511592 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:10.511608 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.512865 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.512870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:10.515601 (+ 2731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:10.515616 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.517218 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.517226 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:10.517474 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:10.517528 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 04:56:10.518285 (+ 757us) client_negotiation.cc:240] Negotiation successful
0809 04:56:10.518420 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12363,"mutex_wait_us":19,"thread_start_us":11654,"threads_started":1}
I20260809 04:56:10.519786 2112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:10.496443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45813 (local address 127.29.243.195:52523)
0809 04:56:10.508900 (+ 12457us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:10.508931 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:10.509181 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:10.510575 (+ 1394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:10.510578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:10.510623 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:10.511055 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:10.511065 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.513221 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.513225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:10.514890 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:10.514909 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.517544 (+ 2635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.517551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:10.517856 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:10.517888 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:56:10.517995 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 04:56:10.518128 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12179,"mutex_wait_us":1,"spinlock_wait_cycles":7552,"thread_start_us":12836,"threads_started":1}
I20260809 04:56:10.520491 2075 raft_consensus.cc:2781] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:10.520593 1347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4a316411c94a8ebf4fda1fc3c59fb4" candidate_uuid: "ffc03d1ddfac40e99eec11793f4784ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a34e4d437d429a84641f1134069a0a" is_pre_election: true
I20260809 04:56:10.520718 2075 raft_consensus.cc:482] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:10.520900 2075 raft_consensus.cc:3037] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:10.523463 2115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:10.496753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:59703 (local address 127.29.243.194:44487)
0809 04:56:10.509883 (+ 13130us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:10.509888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:10.509909 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:56:10.509972 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:10.509976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:10.510041 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:10.510120 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:10.511887 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.512682 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.515768 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.516893 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.517084 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:10.517164 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:10.517712 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 04:56:10.517863 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13021,"thread_start_us":12875,"threads_started":1}
I20260809 04:56:10.524266 1573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4a316411c94a8ebf4fda1fc3c59fb4" candidate_uuid: "ffc03d1ddfac40e99eec11793f4784ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40828a794a945fb9a2b80a3dc16c677" is_pre_election: true
I20260809 04:56:10.524608 1573 raft_consensus.cc:2454] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffc03d1ddfac40e99eec11793f4784ef in term 0.
I20260809 04:56:10.525090 1688 leader_election.cc:302] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d40828a794a945fb9a2b80a3dc16c677, ffc03d1ddfac40e99eec11793f4784ef; no voters:
I20260809 04:56:10.525589 2109 raft_consensus.cc:2781] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:10.525841 2109 raft_consensus.cc:482] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:10.525992 2109 raft_consensus.cc:3037] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:10.605167 2075 raft_consensus.cc:504] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.605175 2109 raft_consensus.cc:504] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.605875 1347 raft_consensus.cc:2379] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ffc03d1ddfac40e99eec11793f4784ef in current term 1: Already voted for candidate 34a34e4d437d429a84641f1134069a0a in this term.
I20260809 04:56:10.606726 2109 leader_election.cc:288] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [CANDIDATE]: Term 1 election: Requested vote from peers 34a34e4d437d429a84641f1134069a0a (127.29.243.193:45813), d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487)
I20260809 04:56:10.606683 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4a316411c94a8ebf4fda1fc3c59fb4" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc03d1ddfac40e99eec11793f4784ef"
I20260809 04:56:10.607053 1812 raft_consensus.cc:2379] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a34e4d437d429a84641f1134069a0a in current term 1: Already voted for candidate ffc03d1ddfac40e99eec11793f4784ef in this term.
I20260809 04:56:10.607187 1573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4a316411c94a8ebf4fda1fc3c59fb4" candidate_uuid: "ffc03d1ddfac40e99eec11793f4784ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40828a794a945fb9a2b80a3dc16c677"
I20260809 04:56:10.607455 1572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4a316411c94a8ebf4fda1fc3c59fb4" candidate_uuid: "34a34e4d437d429a84641f1134069a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40828a794a945fb9a2b80a3dc16c677"
I20260809 04:56:10.607595 1573 raft_consensus.cc:3037] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:10.608129 2075 leader_election.cc:288] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 election: Requested vote from peers d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487), ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113)
I20260809 04:56:10.608552 1232 leader_election.cc:302] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a34e4d437d429a84641f1134069a0a; no voters: d40828a794a945fb9a2b80a3dc16c677, ffc03d1ddfac40e99eec11793f4784ef
I20260809 04:56:10.609267 2075 raft_consensus.cc:2726] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 04:56:10.609097 1347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4a316411c94a8ebf4fda1fc3c59fb4" candidate_uuid: "ffc03d1ddfac40e99eec11793f4784ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a34e4d437d429a84641f1134069a0a"
I20260809 04:56:10.609704 1347 raft_consensus.cc:2379] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ffc03d1ddfac40e99eec11793f4784ef in current term 1: Already voted for candidate 34a34e4d437d429a84641f1134069a0a in this term.
I20260809 04:56:10.611531 1573 raft_consensus.cc:2454] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffc03d1ddfac40e99eec11793f4784ef in term 1.
I20260809 04:56:10.612003 1688 leader_election.cc:302] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d40828a794a945fb9a2b80a3dc16c677, ffc03d1ddfac40e99eec11793f4784ef; no voters: 34a34e4d437d429a84641f1134069a0a
I20260809 04:56:10.612403 2109 raft_consensus.cc:2781] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:10.613067 2109 raft_consensus.cc:686] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 1 LEADER]: Becoming Leader. State: Replica: ffc03d1ddfac40e99eec11793f4784ef, State: Running, Role: LEADER
I20260809 04:56:10.613493 2109 consensus_queue.cc:237] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.617195 1065 catalog_manager.cc:5495] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef reported cstate change: term changed from 0 to 1, leader changed from <none> to ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195). New cstate: current_term: 1 leader_uuid: "ffc03d1ddfac40e99eec11793f4784ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } health_report { overall_health: HEALTHY } } }
I20260809 04:56:10.720695 2134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:10.711664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43324 (local address 127.29.243.195:41113)
0809 04:56:10.712880 (+ 1216us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:10.712885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:10.712930 (+ 45us) server_negotiation.cc:407] Connection header received
0809 04:56:10.713025 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:10.713028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:10.713186 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:10.713389 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:10.713898 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.715140 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.717100 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.717888 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.718027 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:10.718993 (+ 966us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:10.719830 (+ 837us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:10.719959 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:10.720057 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 04:56:10.720129 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1006,"thread_start_us":800,"threads_started":1}
I20260809 04:56:10.742568 2113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:10.734490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:33789 (local address 127.29.243.193:45813)
0809 04:56:10.734709 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:10.734714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:10.735592 (+ 878us) server_negotiation.cc:407] Connection header received
0809 04:56:10.735773 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:10.735777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:10.735846 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:10.735921 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:10.736862 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.737985 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.740416 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.741089 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.741206 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:10.741821 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:10.741971 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:10.742058 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:10.742186 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 04:56:10.742264 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:56:10.743126 2139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:10.734147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45813 (local address 127.29.243.195:33789)
0809 04:56:10.735509 (+ 1362us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:10.735525 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:10.735671 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:10.735972 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:10.735975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:10.735984 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:56:10.736733 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:10.736743 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.738154 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.738158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:10.740277 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:10.740290 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.741364 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.741371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:10.741595 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:10.741656 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:56:10.742074 (+ 418us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:56:10.742087 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 04:56:10.742155 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 04:56:10.742288 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1166,"thread_start_us":890,"threads_started":1}
I20260809 04:56:10.754253 1573 raft_consensus.cc:1261] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Refusing update from remote peer ffc03d1ddfac40e99eec11793f4784ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:10.754381 1347 raft_consensus.cc:1261] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Refusing update from remote peer ffc03d1ddfac40e99eec11793f4784ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:10.755075 2109 consensus_queue.cc:1035] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:10.755540 2116 consensus_queue.cc:1035] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:10.791088 2134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:10.781985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:47007 (local address 127.29.243.195:41113)
0809 04:56:10.782140 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:10.782146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:10.783367 (+ 1221us) server_negotiation.cc:407] Connection header received
0809 04:56:10.783593 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:10.783596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:10.783658 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:10.783727 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:10.784438 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.785998 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.788182 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.789117 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.789242 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:10.790126 (+ 884us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:10.790240 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:10.790352 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:10.790484 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 04:56:10.790566 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 04:56:10.791359 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:10.781925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41113 (local address 127.29.243.193:47007)
0809 04:56:10.783245 (+ 1320us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:10.783297 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:10.783467 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:10.783759 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:10.783763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:10.783774 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:56:10.784251 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:10.784263 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.786185 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.786192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:10.788027 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:10.788043 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:10.789471 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:10.789482 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:10.789860 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:10.789908 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:56:10.790349 (+ 441us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:56:10.790365 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 04:56:10.790453 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:56:10.790603 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1135,"thread_start_us":868,"threads_started":1}
I20260809 04:56:10.832435 1065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:10.832815 1065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:10.836875 1065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5: Sending DeleteTablet for 3 replicas of tablet 2f4a316411c94a8ebf4fda1fc3c59fb4
I20260809 04:56:10.837832 1314 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f4a316411c94a8ebf4fda1fc3c59fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:10 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:58202
I20260809 04:56:10.838069 1538 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f4a316411c94a8ebf4fda1fc3c59fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:10 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:43692
I20260809 04:56:10.838121 1774 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f4a316411c94a8ebf4fda1fc3c59fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:10 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:43312
I20260809 04:56:10.846481 1065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:10.848212 1065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:10.848337 2162 tablet_replica.cc:330] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef: stopping tablet replica
I20260809 04:56:10.848639 2162 raft_consensus.cc:2229] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:10.848984 2160 tablet_replica.cc:330] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a: stopping tablet replica
I20260809 04:56:10.849375 2160 raft_consensus.cc:2229] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:10.849592 2162 raft_consensus.cc:2258] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:10.850108 2160 raft_consensus.cc:2258] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:10.850639 2161 tablet_replica.cc:330] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677: stopping tablet replica
I20260809 04:56:10.850982 2161 raft_consensus.cc:2229] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:10.851207 2162 ts_tablet_manager.cc:1905] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:10.851373 2160 ts_tablet_manager.cc:1905] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:10.851588 2161 raft_consensus.cc:2258] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:10.853600 2161 ts_tablet_manager.cc:1905] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:10.861093 1538 tablet_service.cc:1467] Processing CreateTablet for tablet f69bb426c2d14f698d1d602ee2d11562 (DEFAULT_TABLE table=test-table [id=0008e7821ea5491b82f3da7fc0fa14dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:10.861193 1774 tablet_service.cc:1467] Processing CreateTablet for tablet f69bb426c2d14f698d1d602ee2d11562 (DEFAULT_TABLE table=test-table [id=0008e7821ea5491b82f3da7fc0fa14dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:10.861766 1314 tablet_service.cc:1467] Processing CreateTablet for tablet f69bb426c2d14f698d1d602ee2d11562 (DEFAULT_TABLE table=test-table [id=0008e7821ea5491b82f3da7fc0fa14dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:10.914055 1538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69bb426c2d14f698d1d602ee2d11562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:10.914927 1774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69bb426c2d14f698d1d602ee2d11562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:10.917326 1314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69bb426c2d14f698d1d602ee2d11562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:10.927551 2163 tablet_bootstrap.cc:492] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677: Bootstrap starting.
I20260809 04:56:10.931727 2163 tablet_bootstrap.cc:654] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:10.935799 2168 tablet_bootstrap.cc:492] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a: Bootstrap starting.
I20260809 04:56:10.941191 2168 tablet_bootstrap.cc:654] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:10.941437 2169 tablet_bootstrap.cc:492] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef: Bootstrap starting.
I20260809 04:56:10.942229 2163 tablet_bootstrap.cc:492] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677: No bootstrap required, opened a new log
I20260809 04:56:10.942669 2163 ts_tablet_manager.cc:1397] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.004s
I20260809 04:56:10.944413 2163 raft_consensus.cc:348] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.945041 2163 raft_consensus.cc:374] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:10.945377 2163 raft_consensus.cc:729] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d40828a794a945fb9a2b80a3dc16c677, State: Initialized, Role: FOLLOWER
I20260809 04:56:10.945830 2163 consensus_queue.cc:260] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.947314 2168 tablet_bootstrap.cc:492] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a: No bootstrap required, opened a new log
I20260809 04:56:10.947602 2168 ts_tablet_manager.cc:1397] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260809 04:56:10.948109 2163 ts_tablet_manager.cc:1428] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260809 04:56:10.948454 2169 tablet_bootstrap.cc:654] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:10.949158 2168 raft_consensus.cc:348] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.949671 2168 raft_consensus.cc:374] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:10.950021 2168 raft_consensus.cc:729] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a34e4d437d429a84641f1134069a0a, State: Initialized, Role: FOLLOWER
I20260809 04:56:10.950562 2168 consensus_queue.cc:260] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.951836 2168 ts_tablet_manager.cc:1428] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 04:56:10.955107 2169 tablet_bootstrap.cc:492] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef: No bootstrap required, opened a new log
I20260809 04:56:10.955400 2169 ts_tablet_manager.cc:1397] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260809 04:56:10.957000 2169 raft_consensus.cc:348] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.957314 2169 raft_consensus.cc:374] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:10.957566 2169 raft_consensus.cc:729] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc03d1ddfac40e99eec11793f4784ef, State: Initialized, Role: FOLLOWER
I20260809 04:56:10.957934 2169 consensus_queue.cc:260] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.958827 2169 ts_tablet_manager.cc:1428] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260809 04:56:10.967111 2076 raft_consensus.cc:482] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:10.967495 2076 raft_consensus.cc:504] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:10.968398 2076 leader_election.cc:288] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34a34e4d437d429a84641f1134069a0a (127.29.243.193:45813), ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113)
I20260809 04:56:10.968837 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69bb426c2d14f698d1d602ee2d11562" candidate_uuid: "d40828a794a945fb9a2b80a3dc16c677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc03d1ddfac40e99eec11793f4784ef" is_pre_election: true
I20260809 04:56:10.969046 1347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69bb426c2d14f698d1d602ee2d11562" candidate_uuid: "d40828a794a945fb9a2b80a3dc16c677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a34e4d437d429a84641f1134069a0a" is_pre_election: true
I20260809 04:56:10.969349 1812 raft_consensus.cc:2454] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d40828a794a945fb9a2b80a3dc16c677 in term 0.
I20260809 04:56:10.969465 1347 raft_consensus.cc:2454] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d40828a794a945fb9a2b80a3dc16c677 in term 0.
I20260809 04:56:10.969897 1457 leader_election.cc:302] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d40828a794a945fb9a2b80a3dc16c677, ffc03d1ddfac40e99eec11793f4784ef; no voters:
I20260809 04:56:10.970285 2076 raft_consensus.cc:2781] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:10.970553 2076 raft_consensus.cc:482] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:10.970767 2076 raft_consensus.cc:3037] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:11.016395 2160 ts_tablet_manager.cc:1918] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:11.016415 2161 ts_tablet_manager.cc:1918] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:11.016692 2161 log.cc:1196] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/2f4a316411c94a8ebf4fda1fc3c59fb4
I20260809 04:56:11.016707 2160 log.cc:1196] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/2f4a316411c94a8ebf4fda1fc3c59fb4
I20260809 04:56:11.017495 2161 ts_tablet_manager.cc:1939] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P d40828a794a945fb9a2b80a3dc16c677: Deleting consensus metadata
I20260809 04:56:11.017586 2160 ts_tablet_manager.cc:1939] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P 34a34e4d437d429a84641f1134069a0a: Deleting consensus metadata
I20260809 04:56:11.018105 2162 ts_tablet_manager.cc:1918] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:11.018366 2162 log.cc:1196] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/2f4a316411c94a8ebf4fda1fc3c59fb4
I20260809 04:56:11.019304 1043 catalog_manager.cc:4841] TS 34a34e4d437d429a84641f1134069a0a (127.29.243.193:45813): tablet 2f4a316411c94a8ebf4fda1fc3c59fb4 (table test-table [id=80c70d3960954d9389017a80fbe4b477]) successfully deleted
I20260809 04:56:11.019503 2162 ts_tablet_manager.cc:1939] T 2f4a316411c94a8ebf4fda1fc3c59fb4 P ffc03d1ddfac40e99eec11793f4784ef: Deleting consensus metadata
I20260809 04:56:11.020891 2076 raft_consensus.cc:504] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:11.021116 1046 catalog_manager.cc:4841] TS d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487): tablet 2f4a316411c94a8ebf4fda1fc3c59fb4 (table test-table [id=80c70d3960954d9389017a80fbe4b477]) successfully deleted
I20260809 04:56:11.021664 2076 leader_election.cc:288] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [CANDIDATE]: Term 1 election: Requested vote from peers 34a34e4d437d429a84641f1134069a0a (127.29.243.193:45813), ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113)
I20260809 04:56:11.022367 1347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69bb426c2d14f698d1d602ee2d11562" candidate_uuid: "d40828a794a945fb9a2b80a3dc16c677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a34e4d437d429a84641f1134069a0a"
I20260809 04:56:11.022454 1812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69bb426c2d14f698d1d602ee2d11562" candidate_uuid: "d40828a794a945fb9a2b80a3dc16c677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc03d1ddfac40e99eec11793f4784ef"
I20260809 04:56:11.022708 1812 raft_consensus.cc:3037] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:11.022770 1347 raft_consensus.cc:3037] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:11.068493 1045 catalog_manager.cc:4841] TS ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113): tablet 2f4a316411c94a8ebf4fda1fc3c59fb4 (table test-table [id=80c70d3960954d9389017a80fbe4b477]) successfully deleted
I20260809 04:56:11.119508 1812 raft_consensus.cc:2454] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d40828a794a945fb9a2b80a3dc16c677 in term 1.
I20260809 04:56:11.119525 1347 raft_consensus.cc:2454] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d40828a794a945fb9a2b80a3dc16c677 in term 1.
I20260809 04:56:11.120165 1454 leader_election.cc:302] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a34e4d437d429a84641f1134069a0a, d40828a794a945fb9a2b80a3dc16c677; no voters:
I20260809 04:56:11.120720 2076 raft_consensus.cc:2781] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:11.122720 2076 raft_consensus.cc:686] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 1 LEADER]: Becoming Leader. State: Replica: d40828a794a945fb9a2b80a3dc16c677, State: Running, Role: LEADER
I20260809 04:56:11.123314 2076 consensus_queue.cc:237] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } }
I20260809 04:56:11.126957 1064 catalog_manager.cc:5495] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 reported cstate change: term changed from 0 to 1, leader changed from <none> to d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194). New cstate: current_term: 1 leader_uuid: "d40828a794a945fb9a2b80a3dc16c677" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d40828a794a945fb9a2b80a3dc16c677" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 44487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 } health_report { overall_health: UNKNOWN } } }
I20260809 04:56:11.193001 2178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:11.184815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43722 (local address 127.29.243.194:44487)
0809 04:56:11.186290 (+ 1475us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:11.186295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:11.186336 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:56:11.186400 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:11.186403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:11.186464 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:11.186550 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:11.187065 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:11.188483 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:11.190479 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:11.191284 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:11.191417 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:11.192058 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:11.192199 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:11.192274 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:11.192415 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 04:56:11.192491 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1247,"thread_start_us":877,"threads_started":1}
I20260809 04:56:11.205240 1347 raft_consensus.cc:1261] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Refusing update from remote peer d40828a794a945fb9a2b80a3dc16c677: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:11.205549 1812 raft_consensus.cc:1261] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Refusing update from remote peer d40828a794a945fb9a2b80a3dc16c677: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:11.206138 2175 consensus_queue.cc:1035] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a34e4d437d429a84641f1134069a0a" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:11.206681 2076 consensus_queue.cc:1035] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffc03d1ddfac40e99eec11793f4784ef" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 04:56:11.241959 2122 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 04:56:12.842039 1064 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:12.842351 1064 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:12.848455 1064 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17726e5a5ac04c9b81e0873424d842c5: Sending DeleteTablet for 3 replicas of tablet f69bb426c2d14f698d1d602ee2d11562
I20260809 04:56:12.849308 1538 tablet_service.cc:1514] Processing DeleteTablet for tablet f69bb426c2d14f698d1d602ee2d11562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:12 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:43692
I20260809 04:56:12.849500 1774 tablet_service.cc:1514] Processing DeleteTablet for tablet f69bb426c2d14f698d1d602ee2d11562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:12 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:43312
I20260809 04:56:12.849500 1314 tablet_service.cc:1514] Processing DeleteTablet for tablet f69bb426c2d14f698d1d602ee2d11562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:12 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:58202
I20260809 04:56:12.850243 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1198
I20260809 04:56:12.859362 2217 tablet_replica.cc:330] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a: stopping tablet replica
I20260809 04:56:12.859643 2217 raft_consensus.cc:2229] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:12.860074 2217 raft_consensus.cc:2258] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:12.860334 2215 tablet_replica.cc:330] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677: stopping tablet replica
I20260809 04:56:12.860737 2215 raft_consensus.cc:2229] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:12.861378 2215 raft_consensus.cc:2258] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:12.861512 2217 ts_tablet_manager.cc:1905] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:12.862682 2218 tablet_replica.cc:330] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef: stopping tablet replica
I20260809 04:56:12.863005 2215 ts_tablet_manager.cc:1905] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:12.863014 2218 raft_consensus.cc:2229] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:12.863511 2218 raft_consensus.cc:2258] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:12.864666 2218 ts_tablet_manager.cc:1905] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:12.870635 2217 ts_tablet_manager.cc:1918] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:56:12.870867 2217 log.cc:1196] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/f69bb426c2d14f698d1d602ee2d11562
Aug 09 04:56:12 dist-test-slave-21b9 krb5kdc[996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:56:12.871683 2217 ts_tablet_manager.cc:1939] T f69bb426c2d14f698d1d602ee2d11562 P 34a34e4d437d429a84641f1134069a0a: Deleting consensus metadata
I20260809 04:56:12.871739 2215 ts_tablet_manager.cc:1918] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:56:12.871994 2215 log.cc:1196] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/f69bb426c2d14f698d1d602ee2d11562
I20260809 04:56:12.872715 2215 ts_tablet_manager.cc:1939] T f69bb426c2d14f698d1d602ee2d11562 P d40828a794a945fb9a2b80a3dc16c677: Deleting consensus metadata
I20260809 04:56:12.872913 1043 catalog_manager.cc:4841] TS 34a34e4d437d429a84641f1134069a0a (127.29.243.193:45813): tablet f69bb426c2d14f698d1d602ee2d11562 (table test-table [id=0008e7821ea5491b82f3da7fc0fa14dd]) successfully deleted
I20260809 04:56:12.873898 1046 catalog_manager.cc:4841] TS d40828a794a945fb9a2b80a3dc16c677 (127.29.243.194:44487): tablet f69bb426c2d14f698d1d602ee2d11562 (table test-table [id=0008e7821ea5491b82f3da7fc0fa14dd]) successfully deleted
I20260809 04:56:12.874595 2218 ts_tablet_manager.cc:1918] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:56:12.874814 2218 log.cc:1196] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/f69bb426c2d14f698d1d602ee2d11562
I20260809 04:56:12.875506 2218 ts_tablet_manager.cc:1939] T f69bb426c2d14f698d1d602ee2d11562 P ffc03d1ddfac40e99eec11793f4784ef: Deleting consensus metadata
I20260809 04:56:12.876394 1045 catalog_manager.cc:4841] TS ffc03d1ddfac40e99eec11793f4784ef (127.29.243.195:41113): tablet f69bb426c2d14f698d1d602ee2d11562 (table test-table [id=0008e7821ea5491b82f3da7fc0fa14dd]) successfully deleted
I20260809 04:56:12.881836 2220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:12.852055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58218 (local address 127.29.243.193:45813)
0809 04:56:12.858525 (+ 6470us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:12.858532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:12.858556 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:56:12.858620 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:12.858626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:12.858709 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:12.858816 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:12.860735 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:12.861771 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:12.864243 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:12.865240 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:12.865357 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:12.873464 (+ 8107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:12.873483 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:12.873502 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:12.873600 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:12.878183 (+ 4583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:12.878998 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:12.879007 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:12.879009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:12.879188 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:12.879685 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:12.879689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:12.879691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:12.880238 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:12.880680 (+ 442us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:12.880891 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 04:56:12.881202 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6307,"thread_start_us":6209,"threads_started":1}
W20260809 04:56:12.882692 1421 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58218
W20260809 04:56:12.883527 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1198: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:56:12.883819 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 1198
I20260809 04:56:12.921763 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1432
Aug 09 04:56:12 dist-test-slave-21b9 krb5kdc[996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:56:12.937044 2221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:12.922474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43730 (local address 127.29.243.194:44487)
0809 04:56:12.922944 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:12.922949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:12.922980 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:56:12.923037 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:12.923040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:12.923099 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:12.923186 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:12.925055 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:12.925823 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:12.927119 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:12.927566 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:12.927659 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:12.931591 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:12.931606 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:12.931624 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:12.931663 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:12.934505 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:12.935097 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:12.935103 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:12.935106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:12.935246 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:12.935719 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:12.935722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:12.935724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:12.936099 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:12.936327 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:12.936532 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 04:56:12.936702 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":217,"threads_started":1}
W20260809 04:56:12.937688 1652 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43730
W20260809 04:56:12.938277 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1432: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:56:12.938426 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 1432
I20260809 04:56:12.972820 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1667
Aug 09 04:56:12 dist-test-slave-21b9 krb5kdc[996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:56:12.987622 2222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:12.973671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43330 (local address 127.29.243.195:41113)
0809 04:56:12.974194 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:12.974199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:12.974215 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:56:12.974268 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:12.974272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:12.974330 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:12.974412 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:12.975823 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:12.976637 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:12.977776 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:12.978180 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:12.978272 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:12.982349 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:12.982366 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:12.982385 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:12.982425 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:12.985300 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:12.985904 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:12.985911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:12.985913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:12.986064 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:12.986343 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:12.986346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:12.986348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:12.986684 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:12.986908 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:12.987123 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 04:56:12.987333 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":285,"threads_started":1}
W20260809 04:56:12.988312 1885 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43330
W20260809 04:56:12.988922 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1667: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:56:12.989058 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 1667
I20260809 04:56:13.020144 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1014
I20260809 04:56:13.021704 1147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:13.140408 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 1014
2026-08-09T04:56:13Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9251 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:56:13 dist-test-slave-21b9 krb5kdc[2228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:56:13 dist-test-slave-21b9 krb5kdc[2228](info): set up 2 sockets
Aug 09 04:56:13 dist-test-slave-21b9 krb5kdc[2228](info): commencing operation
krb5kdc: starting...
W20260809 04:56:14.356571 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:56:14 dist-test-slave-21b9 krb5kdc[2228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:56:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:56:14.564216 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44181
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:44181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:14.894563 2249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:14.894852 2249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:14.895004 2249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:14.902220 2249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:14.902352 2249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:14.902433 2249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:14.902521 2249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:14.902585 2249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:14.912405 2249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:44181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:14.913735 2249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:14.915083 2249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:14.927019 2258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:14.927048 2259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:14.927078 2261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:14.927742 2249 server_base.cc:1034] running on GCE node
I20260809 04:56:14.928493 2249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:14.929847 2249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:14.931051 2249 hybrid_clock.cc:648] HybridClock initialized: now 1786251374931005 us; error 44 us; skew 500 ppm
I20260809 04:56:14.931434 2249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:14 dist-test-slave-21b9 krb5kdc[2228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251374, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:14.937541 2249 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:56:14.939036 2249 webserver.cc:460] Webserver started at http://127.29.243.254:38891/ using document root <none> and password file <none>
I20260809 04:56:14.939774 2249 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:14.939895 2249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:14.940218 2249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:14.943207 2249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "f91db94c4f874903b862ef8a8d15bca8"
format_stamp: "Formatted at 2026-08-09 04:56:14 on dist-test-slave-21b9"
server_key: "3ece48516a3f6203fd1ecdb09dfd6c87"
server_key_iv: "19bc1bc82947ea64e0491b3aacfd901a"
server_key_version: "encryptionkey@0"
I20260809 04:56:14.944017 2249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "f91db94c4f874903b862ef8a8d15bca8"
format_stamp: "Formatted at 2026-08-09 04:56:14 on dist-test-slave-21b9"
server_key: "3ece48516a3f6203fd1ecdb09dfd6c87"
server_key_iv: "19bc1bc82947ea64e0491b3aacfd901a"
server_key_version: "encryptionkey@0"
I20260809 04:56:14.949244 2249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:56:14.952780 2273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:14.954169 2249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:56:14.954348 2249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "f91db94c4f874903b862ef8a8d15bca8"
format_stamp: "Formatted at 2026-08-09 04:56:14 on dist-test-slave-21b9"
server_key: "3ece48516a3f6203fd1ecdb09dfd6c87"
server_key_iv: "19bc1bc82947ea64e0491b3aacfd901a"
server_key_version: "encryptionkey@0"
I20260809 04:56:14.954555 2249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:14.990365 2249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:14.994609 2249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:14.994882 2249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:15.013422 2249 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:44181
I20260809 04:56:15.013471 2356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:44181 every 8 connection(s)
I20260809 04:56:15.014844 2249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:15.018846 2359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:15.024233 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 2249
I20260809 04:56:15.024438 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:15.025070 30671 external_mini_cluster.cc:1428] Setting key 14e4627b40154829d734e79ab7d746ad
I20260809 04:56:15.030491 2359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8: Bootstrap starting.
I20260809 04:56:15.034540 2359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:15.035952 2359 log.cc:824] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:15.039191 2359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8: No bootstrap required, opened a new log
Aug 09 04:56:15 dist-test-slave-21b9 krb5kdc[2228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:15.048454 2359 raft_consensus.cc:348] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91db94c4f874903b862ef8a8d15bca8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44181 } }
I20260809 04:56:15.049018 2359 raft_consensus.cc:374] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:15.049161 2359 raft_consensus.cc:729] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f91db94c4f874903b862ef8a8d15bca8, State: Initialized, Role: FOLLOWER
I20260809 04:56:15.049662 2363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:15.027045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55560 (local address 127.29.243.254:44181)
0809 04:56:15.027926 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:15.027938 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:15.027987 (+ 49us) server_negotiation.cc:407] Connection header received
0809 04:56:15.029731 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:15.029756 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:15.030131 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:15.030439 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:15.032533 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:15.033912 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:15.035868 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:15.036748 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:15.036952 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:15.041092 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:15.041100 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:15.041118 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:15.041158 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:15.044654 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:15.045438 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:15.045446 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:15.045453 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:15.045640 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:15.046013 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:15.046018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:15.046022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:15.046655 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:15.046956 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:15.047498 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 04:56:15.047986 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":417,"threads_started":1}
I20260809 04:56:15.049906 2359 consensus_queue.cc:260] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91db94c4f874903b862ef8a8d15bca8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44181 } }
I20260809 04:56:15.050194 2359 raft_consensus.cc:388] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:15.050341 2359 raft_consensus.cc:482] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:15.050518 2359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:15.052896 2359 raft_consensus.cc:504] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91db94c4f874903b862ef8a8d15bca8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44181 } }
I20260809 04:56:15.053372 2359 leader_election.cc:302] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f91db94c4f874903b862ef8a8d15bca8; no voters:
I20260809 04:56:15.053961 2359 leader_election.cc:288] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:15.054080 2366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:15.054768 2366 raft_consensus.cc:686] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [term 1 LEADER]: Becoming Leader. State: Replica: f91db94c4f874903b862ef8a8d15bca8, State: Running, Role: LEADER
I20260809 04:56:15.055253 2366 consensus_queue.cc:237] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91db94c4f874903b862ef8a8d15bca8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44181 } }
I20260809 04:56:15.056032 2359 sys_catalog.cc:564] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:15.058265 2368 sys_catalog.cc:455] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f91db94c4f874903b862ef8a8d15bca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91db94c4f874903b862ef8a8d15bca8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44181 } } }
I20260809 04:56:15.058771 2368 sys_catalog.cc:458] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:15.059098 2370 sys_catalog.cc:455] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f91db94c4f874903b862ef8a8d15bca8. Latest consensus state: current_term: 1 leader_uuid: "f91db94c4f874903b862ef8a8d15bca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91db94c4f874903b862ef8a8d15bca8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44181 } } }
I20260809 04:56:15.059576 2370 sys_catalog.cc:458] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:15.061815 2377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:15.066987 2377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:15.075609 2377 catalog_manager.cc:1347] Generated new cluster ID: a1f1fe9402f84a5f94785bbb0c3b20b3
I20260809 04:56:15.075735 2377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:15.089051 2377 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:15.090401 2377 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:15.098335 2377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f91db94c4f874903b862ef8a8d15bca8: Generated new TSK 0
I20260809 04:56:15.099463 2377 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:56:15.154958 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44181
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:15.502614 2397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:15.502877 2397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:15.502970 2397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:15.510012 2397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:15.510114 2397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:15.510236 2397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:56:15.519769 2397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:15.521013 2397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:15.522441 2397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:15.535246 2405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:15.535400 2409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:15.535341 2406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:15.536392 2397 server_base.cc:1034] running on GCE node
I20260809 04:56:15.536962 2397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:15.537961 2397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:15.539191 2397 hybrid_clock.cc:648] HybridClock initialized: now 1786251375539139 us; error 44 us; skew 500 ppm
I20260809 04:56:15.539534 2397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:15 dist-test-slave-21b9 krb5kdc[2228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251375, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:15.542589 2397 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:56:15.543998 2397 webserver.cc:460] Webserver started at http://127.29.243.193:39769/ using document root <none> and password file <none>
I20260809 04:56:15.544662 2397 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:15.544778 2397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:15.545096 2397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:15.548087 2397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "a098925c5ed940dfa990cc545deb6562"
format_stamp: "Formatted at 2026-08-09 04:56:15 on dist-test-slave-21b9"
server_key: "0daeb1d99327ab1a94c4ce4405698952"
server_key_iv: "45bfb9e0568b57ffcbdb9f8d8709b89d"
server_key_version: "encryptionkey@0"
I20260809 04:56:15.549024 2397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "a098925c5ed940dfa990cc545deb6562"
format_stamp: "Formatted at 2026-08-09 04:56:15 on dist-test-slave-21b9"
server_key: "0daeb1d99327ab1a94c4ce4405698952"
server_key_iv: "45bfb9e0568b57ffcbdb9f8d8709b89d"
server_key_version: "encryptionkey@0"
I20260809 04:56:15.554070 2397 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 04:56:15.557723 2419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:15.559056 2397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:15.559227 2397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "a098925c5ed940dfa990cc545deb6562"
format_stamp: "Formatted at 2026-08-09 04:56:15 on dist-test-slave-21b9"
server_key: "0daeb1d99327ab1a94c4ce4405698952"
server_key_iv: "45bfb9e0568b57ffcbdb9f8d8709b89d"
server_key_version: "encryptionkey@0"
I20260809 04:56:15.559513 2397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:15.586148 2397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:15.590138 2397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:15.590412 2397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:15.591440 2397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:15.592830 2397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:15.592905 2397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:15.593024 2397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:15.593078 2397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:15.623623 2397 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:42389
I20260809 04:56:15.623685 2594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:42389 every 8 connection(s)
I20260809 04:56:15.624910 2397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:56:15.627529 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 2397
I20260809 04:56:15.627729 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:56:15.628355 30671 external_mini_cluster.cc:1428] Setting key 27849bf3b90d8130beeee46e2f43a378
Aug 09 04:56:15 dist-test-slave-21b9 krb5kdc[2228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251375, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:15.647089 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:15.627503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:45939 (local address 127.29.243.254:44181)
0809 04:56:15.631679 (+ 4176us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:15.631685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:15.631701 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:56:15.631752 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:15.631757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:15.631834 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:15.631936 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:15.633785 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:15.634807 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:15.636456 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:15.637005 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:15.637126 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:15.640770 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:15.640776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:15.640779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:15.640813 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:15.643970 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:15.644725 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:15.644731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:15.644735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:15.644903 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:15.645313 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:15.645318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:15.645322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:15.645654 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:15.645791 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:15.646500 (+ 709us) server_negotiation.cc:299] Negotiation successful
0809 04:56:15.646735 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4029,"thread_start_us":3914,"threads_started":1}
I20260809 04:56:15.647943 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:15.628104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44181 (local address 127.29.243.193:45939)
0809 04:56:15.629413 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:15.629453 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:15.631095 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:15.632230 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:15.632242 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:15.632992 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:15.633599 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:15.633619 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:15.634996 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:15.635000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:15.636302 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:15.636315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:15.637207 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:15.637212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:15.637432 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:15.638143 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:15.638167 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:15.640536 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:15.644112 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:15.644120 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:15.644134 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:15.644585 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:15.645011 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:15.645014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:15.645016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:15.645203 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:15.645768 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:15.645774 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:15.646090 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 04:56:15.646322 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 04:56:15.646737 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1043,"thread_start_us":756,"threads_started":1}
I20260809 04:56:15.649662 2596 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44181
I20260809 04:56:15.649988 2596 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:15.650780 2596 heartbeater.cc:511] Master 127.29.243.254:44181 requested a full tablet report, sending...
I20260809 04:56:15.653362 2304 ts_manager.cc:194] Registered new tserver with Master: a098925c5ed940dfa990cc545deb6562 (127.29.243.193:42389)
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
I20260809 04:56:15.655254 2304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:45939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:56:15.682657 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44181
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:16.016037 2606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:16.016397 2606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:16.016566 2606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:16.024169 2606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:16.024288 2606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:16.024452 2606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:56:16.035493 2606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:16.036877 2606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:16.038178 2606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:16.050853 2613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:16.050985 2616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:16.051231 2621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:16.052037 2606 server_base.cc:1034] running on GCE node
I20260809 04:56:16.052690 2606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:16.053673 2606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:16.054881 2606 hybrid_clock.cc:648] HybridClock initialized: now 1786251376054832 us; error 35 us; skew 500 ppm
I20260809 04:56:16.055199 2606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:16 dist-test-slave-21b9 krb5kdc[2228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251376, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:16.058406 2606 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:56:16.059962 2606 webserver.cc:460] Webserver started at http://127.29.243.194:40887/ using document root <none> and password file <none>
I20260809 04:56:16.060642 2606 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:16.060757 2606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:16.061064 2606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:16.064041 2606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "46781d5fbcb440528ca4a1d993d4f153"
format_stamp: "Formatted at 2026-08-09 04:56:16 on dist-test-slave-21b9"
server_key: "8879ec573a8740c42b729631dac73efd"
server_key_iv: "a4f0b2e98333a2ccf43bb03d500a1f4f"
server_key_version: "encryptionkey@0"
I20260809 04:56:16.064852 2606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "46781d5fbcb440528ca4a1d993d4f153"
format_stamp: "Formatted at 2026-08-09 04:56:16 on dist-test-slave-21b9"
server_key: "8879ec573a8740c42b729631dac73efd"
server_key_iv: "a4f0b2e98333a2ccf43bb03d500a1f4f"
server_key_version: "encryptionkey@0"
I20260809 04:56:16.070094 2606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 04:56:16.073666 2632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:16.074958 2606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 04:56:16.075112 2606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "46781d5fbcb440528ca4a1d993d4f153"
format_stamp: "Formatted at 2026-08-09 04:56:16 on dist-test-slave-21b9"
server_key: "8879ec573a8740c42b729631dac73efd"
server_key_iv: "a4f0b2e98333a2ccf43bb03d500a1f4f"
server_key_version: "encryptionkey@0"
I20260809 04:56:16.075313 2606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:16.095867 2606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:16.100142 2606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:16.100411 2606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:16.101413 2606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:16.102856 2606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:16.102955 2606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:16.103075 2606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:16.103127 2606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:16.134115 2606 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:43037
I20260809 04:56:16.134163 2826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:43037 every 8 connection(s)
I20260809 04:56:16.135453 2606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:56:16.143805 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 2606
I20260809 04:56:16.144003 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:56:16.144440 30671 external_mini_cluster.cc:1428] Setting key a253c67d10ad6aee0158bc1bf0ed14d7
Aug 09 04:56:16 dist-test-slave-21b9 krb5kdc[2228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251376, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:16.156083 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:16.137989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:36233 (local address 127.29.243.254:44181)
0809 04:56:16.138168 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:16.138173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:16.139437 (+ 1264us) server_negotiation.cc:407] Connection header received
0809 04:56:16.141370 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:16.141376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:16.141465 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:16.141555 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:16.143024 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:16.143955 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:16.145683 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:16.146274 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:16.146404 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:16.150259 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:16.150264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:16.150268 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:16.150299 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:16.152819 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:16.153679 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:16.153682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:16.153684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:16.153804 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:16.154298 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:16.154302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:16.154304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:16.154616 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:16.154762 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:16.155506 (+ 744us) server_negotiation.cc:299] Negotiation successful
0809 04:56:16.155742 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:56:16.157063 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:16.138492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44181 (local address 127.29.243.194:36233)
0809 04:56:16.139223 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:16.139263 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:16.141112 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:16.141797 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:16.141807 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:16.142304 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:16.142851 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:16.142870 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:16.144128 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:16.144135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:16.145512 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:16.145529 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:16.146513 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:16.146520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:16.146714 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:16.147502 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:16.147530 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:16.150026 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:16.152962 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:16.152973 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:16.152988 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:16.153570 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:16.153905 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:16.153911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:16.153914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:16.154178 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:16.154746 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:16.154753 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:16.155079 (+ 326us) client_negotiation.cc:769] Sending connection context
0809 04:56:16.155384 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 04:56:16.155796 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":259,"threads_started":1}
I20260809 04:56:16.158587 2827 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44181
I20260809 04:56:16.158915 2827 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:16.159726 2827 heartbeater.cc:511] Master 127.29.243.254:44181 requested a full tablet report, sending...
I20260809 04:56:16.161365 2304 ts_manager.cc:194] Registered new tserver with Master: 46781d5fbcb440528ca4a1d993d4f153 (127.29.243.194:43037)
I20260809 04:56:16.162256 2304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:36233
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:56:16.194443 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44181
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:16.525787 2834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:16.526063 2834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:16.526189 2834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:16.533517 2834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:16.533618 2834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:16.533766 2834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:56:16.543368 2834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:16.544644 2834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:16.545962 2834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:16.559024 2840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:16.559002 2842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:16.559131 2845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:16.559857 2834 server_base.cc:1034] running on GCE node
I20260809 04:56:16.560460 2834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:16.561421 2834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:16.562641 2834 hybrid_clock.cc:648] HybridClock initialized: now 1786251376562579 us; error 62 us; skew 500 ppm
I20260809 04:56:16.562994 2834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:16 dist-test-slave-21b9 krb5kdc[2228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251376, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:16.566874 2834 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:56:16.568367 2834 webserver.cc:460] Webserver started at http://127.29.243.195:37719/ using document root <none> and password file <none>
I20260809 04:56:16.569118 2834 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:16.569231 2834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:16.569535 2834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:16.572472 2834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "1004ca769550491d9b88a05e76f20194"
format_stamp: "Formatted at 2026-08-09 04:56:16 on dist-test-slave-21b9"
server_key: "bcf2f5362d5e61f0ee79cbaa57dbaa42"
server_key_iv: "a245cfefd6d00094d77c07bbb5489b22"
server_key_version: "encryptionkey@0"
I20260809 04:56:16.573303 2834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "1004ca769550491d9b88a05e76f20194"
format_stamp: "Formatted at 2026-08-09 04:56:16 on dist-test-slave-21b9"
server_key: "bcf2f5362d5e61f0ee79cbaa57dbaa42"
server_key_iv: "a245cfefd6d00094d77c07bbb5489b22"
server_key_version: "encryptionkey@0"
I20260809 04:56:16.578286 2834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:56:16.581789 2857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:16.583068 2834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:16.583221 2834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "1004ca769550491d9b88a05e76f20194"
format_stamp: "Formatted at 2026-08-09 04:56:16 on dist-test-slave-21b9"
server_key: "bcf2f5362d5e61f0ee79cbaa57dbaa42"
server_key_iv: "a245cfefd6d00094d77c07bbb5489b22"
server_key_version: "encryptionkey@0"
I20260809 04:56:16.583473 2834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:16.611451 2834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:16.615587 2834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:16.615825 2834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:16.616724 2834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:16.618052 2834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:16.618124 2834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:16.618216 2834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:16.618268 2834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:16.649413 2834 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:37501
I20260809 04:56:16.649461 3031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:37501 every 8 connection(s)
I20260809 04:56:16.650727 2834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:56:16.655615 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 2834
I20260809 04:56:16.655793 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:56:16.656358 30671 external_mini_cluster.cc:1428] Setting key 96d8df1c07744bdac453e1807df18068
I20260809 04:56:16.658594 2596 heartbeater.cc:503] Master 127.29.243.254:44181 was elected leader, sending a full tablet report...
Aug 09 04:56:16 dist-test-slave-21b9 krb5kdc[2228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251376, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:16.672449 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:16.653381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:47861 (local address 127.29.243.254:44181)
0809 04:56:16.653572 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:16.653577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:16.654724 (+ 1147us) server_negotiation.cc:407] Connection header received
0809 04:56:16.657172 (+ 2448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:16.657178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:16.657265 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:16.657360 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:16.659677 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:16.660670 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:16.662297 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:16.662758 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:16.662861 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:16.666550 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:16.666566 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:16.666570 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:16.666607 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:16.669088 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:16.669808 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:16.669812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:16.669814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:16.669939 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:16.670459 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:16.670463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:16.670466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:16.670802 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:16.670926 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:16.671842 (+ 916us) server_negotiation.cc:299] Negotiation successful
0809 04:56:16.672082 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:56:16.673475 3040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:16.653809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44181 (local address 127.29.243.195:47861)
0809 04:56:16.654551 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:16.654598 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:16.656839 (+ 2241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:16.657654 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:16.657662 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:16.658375 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:16.659511 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:16.659540 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:16.660811 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:16.660815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:16.662147 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:16.662159 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:16.662985 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:16.662991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:16.663198 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:16.663931 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:16.663955 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:16.666310 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:16.669243 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:16.669251 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:16.669267 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:16.669707 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:16.670045 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:16.670049 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:16.670052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:16.670313 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:16.670945 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:16.670953 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:16.671392 (+ 439us) client_negotiation.cc:769] Sending connection context
0809 04:56:16.671710 (+ 318us) client_negotiation.cc:240] Negotiation successful
0809 04:56:16.672260 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":301,"threads_started":1}
I20260809 04:56:16.674757 3032 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44181
I20260809 04:56:16.675069 3032 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:16.675825 3032 heartbeater.cc:511] Master 127.29.243.254:44181 requested a full tablet report, sending...
I20260809 04:56:16.677201 2303 ts_manager.cc:194] Registered new tserver with Master: 1004ca769550491d9b88a05e76f20194 (127.29.243.195:37501)
I20260809 04:56:16.678033 2303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:47861
I20260809 04:56:16.684762 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:56:16.720291 2598 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:56:16.717855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33866 (local address 127.29.243.254:44181)
0809 04:56:16.718055 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:16.718060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:16.718419 (+ 359us) server_negotiation.cc:407] Connection header received
0809 04:56:16.718539 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:16.718544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:16.718616 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:16.718722 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:16.720033 (+ 1311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33866: BlockingRecv error: recv got EOF from 127.0.0.1:33866 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 04:56:16.724493 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 2397
W20260809 04:56:16.727319 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 2397: Not authorized: Client connection negotiation failed: client connection to 127.29.243.193:42389: server requires authentication, but client does not have Kerberos credentials available
I20260809 04:56:16.727545 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 2397
I20260809 04:56:16.753825 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 2606
W20260809 04:56:16.756640 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 2606: Not authorized: Client connection negotiation failed: client connection to 127.29.243.194:43037: server requires authentication, but client does not have Kerberos credentials available
I20260809 04:56:16.756819 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 2606
I20260809 04:56:16.782135 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 2834
W20260809 04:56:16.785140 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 2834: Not authorized: Client connection negotiation failed: client connection to 127.29.243.195:37501: server requires authentication, but client does not have Kerberos credentials available
I20260809 04:56:16.785300 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 2834
I20260809 04:56:16.809651 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 2249
I20260809 04:56:16.810952 2350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:16.915509 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 2249
2026-08-09T04:56:16Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3771 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:56:17 dist-test-slave-21b9 krb5kdc[3068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:56:17 dist-test-slave-21b9 krb5kdc[3068](info): set up 2 sockets
Aug 09 04:56:17 dist-test-slave-21b9 krb5kdc[3068](info): commencing operation
krb5kdc: starting...
W20260809 04:56:18.145174 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:56:18 dist-test-slave-21b9 krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:56:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:56:18.354339 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:46591
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:46591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:18.683048 3087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:18.683369 3087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:18.683513 3087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:18.691103 3087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:18.691203 3087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:18.691257 3087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:18.691378 3087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:18.691433 3087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:18.701294 3087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:46591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:46591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:18.702603 3087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:18.704059 3087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:18.715634 3097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:18.715783 3101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:18.715762 3104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:18.716650 3087 server_base.cc:1034] running on GCE node
I20260809 04:56:18.717440 3087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:18.718858 3087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:18.720065 3087 hybrid_clock.cc:648] HybridClock initialized: now 1786251378720020 us; error 38 us; skew 500 ppm
I20260809 04:56:18.720415 3087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:18 dist-test-slave-21b9 krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251378, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:18.723505 3087 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:56:18.724944 3087 webserver.cc:460] Webserver started at http://127.29.243.254:37689/ using document root <none> and password file <none>
I20260809 04:56:18.725605 3087 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:18.725726 3087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:18.726040 3087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:18.729040 3087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "f163953df4684c3dabdd0f39d05dd387"
format_stamp: "Formatted at 2026-08-09 04:56:18 on dist-test-slave-21b9"
server_key: "30fd606af8e934b2d45572bc6879e4eb"
server_key_iv: "90c974ebf8fbd951b877ab25c69f1797"
server_key_version: "encryptionkey@0"
I20260809 04:56:18.729811 3087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "f163953df4684c3dabdd0f39d05dd387"
format_stamp: "Formatted at 2026-08-09 04:56:18 on dist-test-slave-21b9"
server_key: "30fd606af8e934b2d45572bc6879e4eb"
server_key_iv: "90c974ebf8fbd951b877ab25c69f1797"
server_key_version: "encryptionkey@0"
I20260809 04:56:18.734982 3087 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 04:56:18.738364 3113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:18.739666 3087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 04:56:18.739831 3087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "f163953df4684c3dabdd0f39d05dd387"
format_stamp: "Formatted at 2026-08-09 04:56:18 on dist-test-slave-21b9"
server_key: "30fd606af8e934b2d45572bc6879e4eb"
server_key_iv: "90c974ebf8fbd951b877ab25c69f1797"
server_key_version: "encryptionkey@0"
I20260809 04:56:18.740013 3087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:18.758077 3087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:18.762009 3087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:18.762271 3087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:18.780720 3087 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:46591
I20260809 04:56:18.780773 3191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:46591 every 8 connection(s)
I20260809 04:56:18.782099 3087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:18.784032 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 3087
I20260809 04:56:18.784271 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:18.784943 30671 external_mini_cluster.cc:1428] Setting key 1ad74a40d2c31e98fe7f58964253cec1
I20260809 04:56:18.786239 3192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:56:18 dist-test-slave-21b9 krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:18.800406 3192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387: Bootstrap starting.
I20260809 04:56:18.804631 3192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:18.805884 3192 log.cc:824] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:18.809247 3192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387: No bootstrap required, opened a new log
I20260809 04:56:18.809502 3196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:18.786758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51782 (local address 127.29.243.254:46591)
0809 04:56:18.787606 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:18.787621 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:18.787671 (+ 50us) server_negotiation.cc:407] Connection header received
0809 04:56:18.789696 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:18.789726 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:18.790276 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:18.790729 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:18.792161 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:18.794280 (+ 2119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:18.795640 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:18.796658 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:18.796849 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:18.801090 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:18.801101 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:18.801122 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:18.801167 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:18.804956 (+ 3789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:18.805698 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:18.805708 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:18.805717 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:18.805928 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:18.806334 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:18.806340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:18.806343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:18.806990 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:18.807253 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:18.807752 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 04:56:18.808190 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":341,"threads_started":1}
I20260809 04:56:18.813560 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:56:18.817332 3192 raft_consensus.cc:348] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163953df4684c3dabdd0f39d05dd387" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46591 } }
I20260809 04:56:18.817672 3192 raft_consensus.cc:374] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:18.817772 3192 raft_consensus.cc:729] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f163953df4684c3dabdd0f39d05dd387, State: Initialized, Role: FOLLOWER
I20260809 04:56:18.818370 3192 consensus_queue.cc:260] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163953df4684c3dabdd0f39d05dd387" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46591 } }
I20260809 04:56:18.818605 3192 raft_consensus.cc:388] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:18.818742 3192 raft_consensus.cc:482] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:18.818898 3192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:18.821049 3192 raft_consensus.cc:504] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163953df4684c3dabdd0f39d05dd387" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46591 } }
I20260809 04:56:18.821537 3192 leader_election.cc:302] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f163953df4684c3dabdd0f39d05dd387; no voters:
I20260809 04:56:18.822113 3192 leader_election.cc:288] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:18.822273 3198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:18.823427 3198 raft_consensus.cc:686] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [term 1 LEADER]: Becoming Leader. State: Replica: f163953df4684c3dabdd0f39d05dd387, State: Running, Role: LEADER
I20260809 04:56:18.823935 3198 consensus_queue.cc:237] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163953df4684c3dabdd0f39d05dd387" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46591 } }
I20260809 04:56:18.824427 3192 sys_catalog.cc:564] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:18.826498 3201 sys_catalog.cc:455] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f163953df4684c3dabdd0f39d05dd387. Latest consensus state: current_term: 1 leader_uuid: "f163953df4684c3dabdd0f39d05dd387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163953df4684c3dabdd0f39d05dd387" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46591 } } }
I20260809 04:56:18.826887 3201 sys_catalog.cc:458] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:18.827497 3200 sys_catalog.cc:455] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f163953df4684c3dabdd0f39d05dd387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163953df4684c3dabdd0f39d05dd387" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46591 } } }
I20260809 04:56:18.827791 3200 sys_catalog.cc:458] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:18.829545 3209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:18.835206 3209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:18.854409 3209 catalog_manager.cc:1347] Generated new cluster ID: 0182324ab6d9453b98be573c7d8b1bba
I20260809 04:56:18.854557 3209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:18.868569 3196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:18.860896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51786 (local address 127.29.243.254:46591)
0809 04:56:18.861110 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:18.861117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:18.861697 (+ 580us) server_negotiation.cc:407] Connection header received
0809 04:56:18.861884 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:18.861888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:18.861958 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:18.862049 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:18.863138 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:18.863938 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:18.865222 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:18.865673 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:18.865802 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:18.866857 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:18.866862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:18.866865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:18.867761 (+ 896us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:18.867809 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:18.867837 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:18.867945 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:18.868081 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 04:56:18.868180 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 04:56:18.874950 3209 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:18.876299 3209 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:18.884024 3209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f163953df4684c3dabdd0f39d05dd387: Generated new TSK 0
I20260809 04:56:18.884855 3209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:56:18.894357 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 3087
I20260809 04:56:18.896026 3186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:18.985941 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 3087
2026-08-09T04:56:19Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2064 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:56:19 dist-test-slave-21b9 krb5kdc[3247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:56:19 dist-test-slave-21b9 krb5kdc[3247](info): set up 2 sockets
Aug 09 04:56:19 dist-test-slave-21b9 krb5kdc[3247](info): commencing operation
krb5kdc: starting...
W20260809 04:56:20.192763 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:56:20 dist-test-slave-21b9 krb5kdc[3247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:56:20.404337 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35111
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:35111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:20.748253 3264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:20.748566 3264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:20.748713 3264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:20.756064 3264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:20.756186 3264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:20.756269 3264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:20.756368 3264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:20.756434 3264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:20.766494 3264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:35111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:20.767942 3264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:20.769412 3264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:20.781633 3274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:20.781685 3279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:20.781826 3284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:20.782491 3264 server_base.cc:1034] running on GCE node
I20260809 04:56:20.783301 3264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:20.784739 3264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:20.785964 3264 hybrid_clock.cc:648] HybridClock initialized: now 1786251380785909 us; error 34 us; skew 500 ppm
I20260809 04:56:20.786355 3264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:20 dist-test-slave-21b9 krb5kdc[3247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251380, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:20.792631 3264 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:56:20.794052 3264 webserver.cc:460] Webserver started at http://127.29.243.254:36437/ using document root <none> and password file <none>
I20260809 04:56:20.794713 3264 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:20.794816 3264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:20.795086 3264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:20.798166 3264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "848a163d873043c791a61c815500dc08"
format_stamp: "Formatted at 2026-08-09 04:56:20 on dist-test-slave-21b9"
server_key: "4e8bfbc7b47944b176bdab37345c990a"
server_key_iv: "a610251fc0e562538d46ab0ddd81ba58"
server_key_version: "encryptionkey@0"
I20260809 04:56:20.798893 3264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "848a163d873043c791a61c815500dc08"
format_stamp: "Formatted at 2026-08-09 04:56:20 on dist-test-slave-21b9"
server_key: "4e8bfbc7b47944b176bdab37345c990a"
server_key_iv: "a610251fc0e562538d46ab0ddd81ba58"
server_key_version: "encryptionkey@0"
I20260809 04:56:20.804113 3264 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 04:56:20.807562 3292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:20.808921 3264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:56:20.809088 3264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "848a163d873043c791a61c815500dc08"
format_stamp: "Formatted at 2026-08-09 04:56:20 on dist-test-slave-21b9"
server_key: "4e8bfbc7b47944b176bdab37345c990a"
server_key_iv: "a610251fc0e562538d46ab0ddd81ba58"
server_key_version: "encryptionkey@0"
I20260809 04:56:20.809257 3264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:20.827735 3264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:20.831936 3264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:20.832162 3264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:20.850049 3264 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:35111
I20260809 04:56:20.850081 3374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:35111 every 8 connection(s)
I20260809 04:56:20.851413 3264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:20.853097 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 3264
I20260809 04:56:20.853348 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:20.854176 30671 external_mini_cluster.cc:1428] Setting key 64a1d1ed9e536e9b5c97811d1e76b320
I20260809 04:56:20.856154 3375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:56:20 dist-test-slave-21b9 krb5kdc[3247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:20.869828 3375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08: Bootstrap starting.
I20260809 04:56:20.873361 3375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:20.874650 3375 log.cc:824] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:20.877964 3375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08: No bootstrap required, opened a new log
I20260809 04:56:20.878189 3379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:20.856284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47960 (local address 127.29.243.254:35111)
0809 04:56:20.857089 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:20.857100 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:20.857142 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:56:20.858697 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:20.858719 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:20.859093 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:20.859477 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:20.861250 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:20.863058 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:20.864739 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:20.865639 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:20.865795 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:20.870691 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:20.870700 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:20.870716 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:20.870759 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:20.873857 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:20.874674 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:20.874681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:20.874688 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:20.874884 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:20.875336 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:20.875346 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:20.875348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:20.876125 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:20.876364 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:20.876794 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 04:56:20.877156 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":343,"threads_started":1}
I20260809 04:56:20.885330 3375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848a163d873043c791a61c815500dc08" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35111 } }
I20260809 04:56:20.885694 3375 raft_consensus.cc:374] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:20.885803 3375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848a163d873043c791a61c815500dc08, State: Initialized, Role: FOLLOWER
I20260809 04:56:20.886402 3375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848a163d873043c791a61c815500dc08" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35111 } }
I20260809 04:56:20.886623 3375 raft_consensus.cc:388] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:20.886729 3375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:20.886868 3375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:20.888967 3375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848a163d873043c791a61c815500dc08" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35111 } }
I20260809 04:56:20.889441 3375 leader_election.cc:302] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 848a163d873043c791a61c815500dc08; no voters:
I20260809 04:56:20.889938 3375 leader_election.cc:288] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:20.890072 3383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:20.891227 3383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [term 1 LEADER]: Becoming Leader. State: Replica: 848a163d873043c791a61c815500dc08, State: Running, Role: LEADER
I20260809 04:56:20.891774 3383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848a163d873043c791a61c815500dc08" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35111 } }
I20260809 04:56:20.892184 3375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:20.894572 3385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 848a163d873043c791a61c815500dc08. Latest consensus state: current_term: 1 leader_uuid: "848a163d873043c791a61c815500dc08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848a163d873043c791a61c815500dc08" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35111 } } }
I20260809 04:56:20.894554 3384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "848a163d873043c791a61c815500dc08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848a163d873043c791a61c815500dc08" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35111 } } }
I20260809 04:56:20.894958 3385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:20.894958 3384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:20.898445 3396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:20.903857 3396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:20.912720 3396 catalog_manager.cc:1347] Generated new cluster ID: 7d7635487a4941e598d79e0129333388
I20260809 04:56:20.912847 3396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:20.943465 3396 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:20.945488 3396 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:20.956507 3396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 848a163d873043c791a61c815500dc08: Generated new TSK 0
I20260809 04:56:20.957587 3396 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:56:21.013159 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35111
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:21.357085 3422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:21.357381 3422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:21.357520 3422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:21.364681 3422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:21.364807 3422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:21.364961 3422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:56:21.374701 3422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:21.376157 3422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:21.377525 3422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:21.390551 3432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:21.390641 3435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:21.390614 3428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:21.391209 3422 server_base.cc:1034] running on GCE node
I20260809 04:56:21.391814 3422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:21.392776 3422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:21.393994 3422 hybrid_clock.cc:648] HybridClock initialized: now 1786251381393936 us; error 50 us; skew 500 ppm
I20260809 04:56:21.394304 3422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:21 dist-test-slave-21b9 krb5kdc[3247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251381, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:21.397542 3422 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:56:21.399015 3422 webserver.cc:460] Webserver started at http://127.29.243.193:37803/ using document root <none> and password file <none>
I20260809 04:56:21.399731 3422 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:21.399837 3422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:21.400113 3422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:21.403056 3422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "f53fb17ce0da42b6992b9e88019dbc94"
format_stamp: "Formatted at 2026-08-09 04:56:21 on dist-test-slave-21b9"
server_key: "7c723fe299a3538d897f4dd947598f50"
server_key_iv: "cd3a6bba0fa2441ddba300f9ebcae366"
server_key_version: "encryptionkey@0"
I20260809 04:56:21.403846 3422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "f53fb17ce0da42b6992b9e88019dbc94"
format_stamp: "Formatted at 2026-08-09 04:56:21 on dist-test-slave-21b9"
server_key: "7c723fe299a3538d897f4dd947598f50"
server_key_iv: "cd3a6bba0fa2441ddba300f9ebcae366"
server_key_version: "encryptionkey@0"
I20260809 04:56:21.408892 3422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:56:21.412402 3445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:21.413791 3422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:21.413930 3422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "f53fb17ce0da42b6992b9e88019dbc94"
format_stamp: "Formatted at 2026-08-09 04:56:21 on dist-test-slave-21b9"
server_key: "7c723fe299a3538d897f4dd947598f50"
server_key_iv: "cd3a6bba0fa2441ddba300f9ebcae366"
server_key_version: "encryptionkey@0"
I20260809 04:56:21.414109 3422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:21.448905 3422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:21.453284 3422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:21.453575 3422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:21.454605 3422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:21.456269 3422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:21.456362 3422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:21.456472 3422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:21.456516 3422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:21.487098 3422 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:41405
I20260809 04:56:21.487144 3623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:41405 every 8 connection(s)
I20260809 04:56:21.488431 3422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:56:21.497994 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 3422
I20260809 04:56:21.498224 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:56:21.498899 30671 external_mini_cluster.cc:1428] Setting key 565815c8b38979a7a35567f36d73a57a
Aug 09 04:56:21 dist-test-slave-21b9 krb5kdc[3247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251381, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:21.508370 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:21.491058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:41673 (local address 127.29.243.254:35111)
0809 04:56:21.491534 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:21.491539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:21.492514 (+ 975us) server_negotiation.cc:407] Connection header received
0809 04:56:21.494458 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:21.494464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:21.494549 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:21.494649 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:21.496219 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:21.496955 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:21.498579 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:21.498981 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:21.499095 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:21.502709 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:21.502714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:21.502718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:21.502751 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:21.505292 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:21.506031 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:21.506036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:21.506038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:21.506161 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:21.506597 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:21.506601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:21.506603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:21.506878 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:21.507014 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:21.507780 (+ 766us) server_negotiation.cc:299] Negotiation successful
0809 04:56:21.508017 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":263,"threads_started":1}
I20260809 04:56:21.509394 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:21.491595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35111 (local address 127.29.243.193:41673)
0809 04:56:21.492357 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:21.492397 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:21.494189 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:21.494902 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:21.494911 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:21.495486 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:21.496044 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:21.496062 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:21.497114 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:21.497117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:21.498430 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:21.498455 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:21.499258 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:21.499296 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:21.499602 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:21.500258 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:21.500286 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:21.502499 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:21.505442 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:21.505451 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:21.505469 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:21.505890 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:21.506267 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:21.506272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:21.506276 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:21.506480 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:21.507006 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:21.507015 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:21.507390 (+ 375us) client_negotiation.cc:769] Sending connection context
0809 04:56:21.507643 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 04:56:21.508200 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":301,"threads_started":1}
I20260809 04:56:21.511083 3624 heartbeater.cc:347] Connected to a master server at 127.29.243.254:35111
I20260809 04:56:21.511447 3624 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:21.512244 3624 heartbeater.cc:511] Master 127.29.243.254:35111 requested a full tablet report, sending...
I20260809 04:56:21.514825 3316 ts_manager.cc:194] Registered new tserver with Master: f53fb17ce0da42b6992b9e88019dbc94 (127.29.243.193:41405)
I20260809 04:56:21.516811 3316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:41673
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:56:21.550906 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35111
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:21.888907 3640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:21.889230 3640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:21.889328 3640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:21.896574 3640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:21.896683 3640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:21.896790 3640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:56:21.906351 3640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:21.907686 3640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:21.908968 3640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:21.921948 3648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:21.921978 3651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:21.922199 3655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:21.923486 3640 server_base.cc:1034] running on GCE node
I20260809 04:56:21.924095 3640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:21.925019 3640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:21.926246 3640 hybrid_clock.cc:648] HybridClock initialized: now 1786251381926187 us; error 47 us; skew 500 ppm
I20260809 04:56:21.926601 3640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:21 dist-test-slave-21b9 krb5kdc[3247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251381, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:21.929705 3640 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:56:21.931023 3640 webserver.cc:460] Webserver started at http://127.29.243.194:35707/ using document root <none> and password file <none>
I20260809 04:56:21.931761 3640 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:21.931864 3640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:21.932119 3640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:21.934979 3640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "8fb254f40df14a1ba93c43fc9315af78"
format_stamp: "Formatted at 2026-08-09 04:56:21 on dist-test-slave-21b9"
server_key: "7d3db3739e5ca7f41a6fd04ddec7d443"
server_key_iv: "0f3dcbf78a6fcb3f07aefbc4432b343a"
server_key_version: "encryptionkey@0"
I20260809 04:56:21.935745 3640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "8fb254f40df14a1ba93c43fc9315af78"
format_stamp: "Formatted at 2026-08-09 04:56:21 on dist-test-slave-21b9"
server_key: "7d3db3739e5ca7f41a6fd04ddec7d443"
server_key_iv: "0f3dcbf78a6fcb3f07aefbc4432b343a"
server_key_version: "encryptionkey@0"
I20260809 04:56:21.940672 3640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:56:21.944056 3665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:21.945502 3640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:56:21.945665 3640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "8fb254f40df14a1ba93c43fc9315af78"
format_stamp: "Formatted at 2026-08-09 04:56:21 on dist-test-slave-21b9"
server_key: "7d3db3739e5ca7f41a6fd04ddec7d443"
server_key_iv: "0f3dcbf78a6fcb3f07aefbc4432b343a"
server_key_version: "encryptionkey@0"
I20260809 04:56:21.945847 3640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:21.959096 3640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:21.962854 3640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:21.963145 3640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:21.964061 3640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:21.965330 3640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:21.965407 3640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:21.965499 3640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:21.965543 3640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:21.995507 3640 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:45439
I20260809 04:56:21.995555 3826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:45439 every 8 connection(s)
I20260809 04:56:21.996771 3640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:56:22.003964 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 3640
I20260809 04:56:22.004163 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:56:22.004745 30671 external_mini_cluster.cc:1428] Setting key 57179959b4768dde3045fa67f4edfe69
Aug 09 04:56:22 dist-test-slave-21b9 krb5kdc[3247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251381, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:22.019732 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:21.999359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:39273 (local address 127.29.243.254:35111)
0809 04:56:21.999571 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:21.999577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:22.000791 (+ 1214us) server_negotiation.cc:407] Connection header received
0809 04:56:22.002558 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:22.002563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:22.002636 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:22.002730 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:22.004314 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.005371 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.007110 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.007785 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.007908 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:22.012159 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:22.012167 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:22.012172 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:22.012218 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:22.015676 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.016622 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.016628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.016631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.016809 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.017512 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.017518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.017521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.017886 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:22.018317 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:22.019245 (+ 928us) server_negotiation.cc:299] Negotiation successful
0809 04:56:22.019488 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:56:22.021098 3833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:21.999913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35111 (local address 127.29.243.194:39273)
0809 04:56:22.000633 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:22.000671 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:22.002278 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:22.002926 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:22.002935 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:22.003508 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:22.004091 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:22.004113 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.005561 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.005567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:22.006967 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:22.006980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.007983 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.007988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:22.008186 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:22.008809 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:22.008832 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:22.011877 (+ 3045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:22.015845 (+ 3968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:22.015856 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:22.015874 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:22.016472 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:22.016950 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:22.016955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:22.016959 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:22.017342 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:22.018391 (+ 1049us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:22.018400 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:22.018835 (+ 435us) client_negotiation.cc:769] Sending connection context
0809 04:56:22.019131 (+ 296us) client_negotiation.cc:240] Negotiation successful
0809 04:56:22.019758 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":268,"threads_started":1}
I20260809 04:56:22.022761 3831 heartbeater.cc:347] Connected to a master server at 127.29.243.254:35111
I20260809 04:56:22.023113 3831 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:22.024022 3831 heartbeater.cc:511] Master 127.29.243.254:35111 requested a full tablet report, sending...
I20260809 04:56:22.025848 3316 ts_manager.cc:194] Registered new tserver with Master: 8fb254f40df14a1ba93c43fc9315af78 (127.29.243.194:45439)
I20260809 04:56:22.026960 3316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:39273
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:56:22.056588 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35111
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:22.385640 3837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:22.385929 3837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:22.386058 3837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:22.393329 3837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:22.393460 3837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:22.393621 3837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:56:22.403365 3837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:22.404708 3837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:22.406090 3837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:22.419366 3845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:22.419299 3844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:22.419780 3847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:22.420341 3837 server_base.cc:1034] running on GCE node
I20260809 04:56:22.420943 3837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:22.421842 3837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:22.423056 3837 hybrid_clock.cc:648] HybridClock initialized: now 1786251382423003 us; error 44 us; skew 500 ppm
I20260809 04:56:22.423413 3837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:22 dist-test-slave-21b9 krb5kdc[3247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251382, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:22.426554 3837 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:56:22.428069 3837 webserver.cc:460] Webserver started at http://127.29.243.195:42695/ using document root <none> and password file <none>
I20260809 04:56:22.428768 3837 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:22.428889 3837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:22.429208 3837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:22.432137 3837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "ac0e9d2dab4843c6819bccd1b645619f"
format_stamp: "Formatted at 2026-08-09 04:56:22 on dist-test-slave-21b9"
server_key: "1155a7a652fb7efad7fb490be44bb819"
server_key_iv: "0433f66414376ed506fced7a378b16e4"
server_key_version: "encryptionkey@0"
I20260809 04:56:22.432901 3837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "ac0e9d2dab4843c6819bccd1b645619f"
format_stamp: "Formatted at 2026-08-09 04:56:22 on dist-test-slave-21b9"
server_key: "1155a7a652fb7efad7fb490be44bb819"
server_key_iv: "0433f66414376ed506fced7a378b16e4"
server_key_version: "encryptionkey@0"
I20260809 04:56:22.438074 3837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:56:22.441612 3854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:22.442965 3837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 04:56:22.443130 3837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "ac0e9d2dab4843c6819bccd1b645619f"
format_stamp: "Formatted at 2026-08-09 04:56:22 on dist-test-slave-21b9"
server_key: "1155a7a652fb7efad7fb490be44bb819"
server_key_iv: "0433f66414376ed506fced7a378b16e4"
server_key_version: "encryptionkey@0"
I20260809 04:56:22.443359 3837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:22.459398 3837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:22.463546 3837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:22.463799 3837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:22.464748 3837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:22.466121 3837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:22.466193 3837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:22.466303 3837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:22.466346 3837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:22.497545 3837 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:44135
I20260809 04:56:22.497635 4029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:44135 every 8 connection(s)
I20260809 04:56:22.498862 3837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:56:22.507671 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 3837
I20260809 04:56:22.507889 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:56:22.508438 30671 external_mini_cluster.cc:1428] Setting key 3b7f8d8c78d154d0fdd16321ce619233
Aug 09 04:56:22 dist-test-slave-21b9 krb5kdc[3247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251382, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:22.518941 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:22.501364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:35465 (local address 127.29.243.254:35111)
0809 04:56:22.501532 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:22.501537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:22.502727 (+ 1190us) server_negotiation.cc:407] Connection header received
0809 04:56:22.504560 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:22.504566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:22.504677 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:22.504766 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:22.506240 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.506991 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.509204 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.509704 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.509816 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:22.513407 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:22.513413 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:22.513418 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:22.513451 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:22.515952 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.516641 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.516645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.516648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.516784 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.517223 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.517226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.517228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.517519 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:22.517697 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:22.518362 (+ 665us) server_negotiation.cc:299] Negotiation successful
0809 04:56:22.518579 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:56:22.519886 4033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:22.501818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35111 (local address 127.29.243.195:35465)
0809 04:56:22.502559 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:22.502603 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:22.504286 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:22.505006 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:22.505014 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:22.505519 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:22.506069 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:22.506086 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.507151 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.507155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:22.509051 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:22.509079 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.509925 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.509932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:22.510164 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:22.510819 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:22.510842 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:22.513208 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:22.516082 (+ 2874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:22.516092 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:22.516109 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:22.516520 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:22.516894 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:22.516899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:22.516903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:22.517089 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:22.517664 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:22.517671 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:22.518002 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 04:56:22.518231 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 04:56:22.518773 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":270,"threads_started":1}
I20260809 04:56:22.520295 3624 heartbeater.cc:503] Master 127.29.243.254:35111 was elected leader, sending a full tablet report...
I20260809 04:56:22.521365 4031 heartbeater.cc:347] Connected to a master server at 127.29.243.254:35111
I20260809 04:56:22.521641 4031 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:22.522476 4031 heartbeater.cc:511] Master 127.29.243.254:35111 requested a full tablet report, sending...
I20260809 04:56:22.523801 3316 ts_manager.cc:194] Registered new tserver with Master: ac0e9d2dab4843c6819bccd1b645619f (127.29.243.195:44135)
I20260809 04:56:22.524562 3316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:35465
I20260809 04:56:22.535738 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:56:22 dist-test-slave-21b9 krb5kdc[3247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251382, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 04:56:22 dist-test-slave-21b9 krb5kdc[3247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251382, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:22.589111 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:22.574790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47974 (local address 127.29.243.254:35111)
0809 04:56:22.575030 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:22.575037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:22.575397 (+ 360us) server_negotiation.cc:407] Connection header received
0809 04:56:22.575651 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:22.575655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:22.575728 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:22.575825 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:22.577212 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.577969 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.579227 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.579775 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.579898 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:22.584016 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:22.584023 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:22.584028 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:22.584076 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:22.586870 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.587533 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.587539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.587542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.587681 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.587981 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.587984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.587986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.588296 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:22.588517 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:22.588687 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 04:56:22.588849 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 04:56:22.589635 3316 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47974
I20260809 04:56:22.590967 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 3422
Aug 09 04:56:22 dist-test-slave-21b9 krb5kdc[3247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251382, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:56:22.605934 4045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:22.592122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50910 (local address 127.29.243.193:41405)
0809 04:56:22.592781 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:22.592788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:22.592814 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:56:22.592870 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:22.592884 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:22.593101 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:22.593220 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:22.594299 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.595353 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.596459 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.597064 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.597161 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:22.600867 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:22.600890 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:22.600906 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:22.600942 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:22.603656 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.604244 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.604251 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.604253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.604389 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.604683 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.604686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.604688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.605044 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:22.605284 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:22.605450 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 04:56:22.605640 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":314,"threads_started":1}
W20260809 04:56:22.607172 3618 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50910
W20260809 04:56:22.608359 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 3422: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:56:22.608573 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 3422
I20260809 04:56:22.663245 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 3640
Aug 09 04:56:22 dist-test-slave-21b9 krb5kdc[3247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251382, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:56:22.679349 4048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:22.664428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50658 (local address 127.29.243.194:45439)
0809 04:56:22.665008 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:22.665016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:22.665044 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:56:22.665109 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:22.665115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:22.665333 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:22.665468 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:22.666755 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.667831 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.669160 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.669761 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.669864 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:22.673964 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:22.673977 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:22.673994 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:22.674039 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:22.676917 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.677514 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.677522 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.677524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.677658 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.677922 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.677925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.677927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.678275 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:22.678515 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:22.678730 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 04:56:22.678940 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":252,"threads_started":1}
W20260809 04:56:22.680716 3821 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50658
W20260809 04:56:22.681816 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 3640: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:56:22.681991 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 3640
I20260809 04:56:22.707118 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 3837
Aug 09 04:56:22 dist-test-slave-21b9 krb5kdc[3247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251382, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:56:22.723598 4049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:22.708234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.29.243.195:44135)
0809 04:56:22.708910 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:22.708918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:22.708948 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:56:22.709013 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:22.709020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:22.709251 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:22.709394 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:22.710719 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.711872 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.713212 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:22.713775 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:22.713880 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:22.718160 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:22.718172 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:22.718189 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:22.718231 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:22.721181 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.721791 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.721799 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.721802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.721936 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:22.722218 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:22.722221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:22.722223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:22.722582 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:22.722812 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:22.723004 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 04:56:22.723212 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":340,"threads_started":1}
W20260809 04:56:22.724983 4022 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60084
W20260809 04:56:22.726063 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 3837: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:56:22.726209 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 3837
I20260809 04:56:22.748911 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 3264
I20260809 04:56:22.750156 3368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:22.854947 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 3264
2026-08-09T04:56:22Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3875 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:56:22 dist-test-slave-21b9 krb5kdc[4056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:56:22 dist-test-slave-21b9 krb5kdc[4056](info): set up 2 sockets
Aug 09 04:56:22 dist-test-slave-21b9 krb5kdc[4056](info): commencing operation
krb5kdc: starting...
W20260809 04:56:24.064841 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:56:24 dist-test-slave-21b9 krb5kdc[4056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:56:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:56:24.286441 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:43283
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:43283
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:24.619419 4075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:24.619716 4075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:24.619815 4075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:24.626902 4075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:24.627002 4075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:24.627056 4075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:24.627112 4075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:24.627157 4075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:24.636739 4075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:43283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:43283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:24.638036 4075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:24.639312 4075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:24.650890 4090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:24.650944 4087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:24.650979 4085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:24.652716 4075 server_base.cc:1034] running on GCE node
I20260809 04:56:24.653461 4075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:24.654999 4075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:24.656200 4075 hybrid_clock.cc:648] HybridClock initialized: now 1786251384656154 us; error 35 us; skew 500 ppm
I20260809 04:56:24.656517 4075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:24 dist-test-slave-21b9 krb5kdc[4056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251384, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:24.659641 4075 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:56:24.661113 4075 webserver.cc:460] Webserver started at http://127.29.243.254:42039/ using document root <none> and password file <none>
I20260809 04:56:24.661775 4075 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:24.661876 4075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:24.662237 4075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:24.665426 4075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "ff11805eb237495690d8d3e9c30e2457"
format_stamp: "Formatted at 2026-08-09 04:56:24 on dist-test-slave-21b9"
server_key: "aa34ecb288e33bc08d4b245e0af78737"
server_key_iv: "de786b43b225154b4be0e25c7fe50a6a"
server_key_version: "encryptionkey@0"
I20260809 04:56:24.666196 4075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "ff11805eb237495690d8d3e9c30e2457"
format_stamp: "Formatted at 2026-08-09 04:56:24 on dist-test-slave-21b9"
server_key: "aa34ecb288e33bc08d4b245e0af78737"
server_key_iv: "de786b43b225154b4be0e25c7fe50a6a"
server_key_version: "encryptionkey@0"
I20260809 04:56:24.671612 4075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 04:56:24.675084 4099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:24.676496 4075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:24.676646 4075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "ff11805eb237495690d8d3e9c30e2457"
format_stamp: "Formatted at 2026-08-09 04:56:24 on dist-test-slave-21b9"
server_key: "aa34ecb288e33bc08d4b245e0af78737"
server_key_iv: "de786b43b225154b4be0e25c7fe50a6a"
server_key_version: "encryptionkey@0"
I20260809 04:56:24.676820 4075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:24.693357 4075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:24.697414 4075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:24.697660 4075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:24.716061 4075 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:43283
I20260809 04:56:24.716084 4184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:43283 every 8 connection(s)
I20260809 04:56:24.717435 4075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:24.718755 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 4075
I20260809 04:56:24.718971 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:24.719883 30671 external_mini_cluster.cc:1428] Setting key 801ec698a2c911eaa7610e7420ddad1d
I20260809 04:56:24.722404 4186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:56:24 dist-test-slave-21b9 krb5kdc[4056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:24.733397 4186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457: Bootstrap starting.
I20260809 04:56:24.736799 4186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:24.737933 4186 log.cc:824] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:24.741928 4186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457: No bootstrap required, opened a new log
I20260809 04:56:24.742789 4192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:24.722032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57946 (local address 127.29.243.254:43283)
0809 04:56:24.722807 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:24.722820 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:24.722863 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:56:24.724551 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:24.724575 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:24.724947 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:24.725325 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:24.726829 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:24.728214 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:24.729451 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:24.730148 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:24.730328 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:24.734434 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:24.734441 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:24.734456 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:24.734507 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:24.737563 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:24.738379 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:24.738388 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:24.738395 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:24.739657 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:24.740057 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:24.740061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:24.740063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:24.740600 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:24.740887 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:24.741359 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 04:56:24.741758 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":331,"threads_started":1}
I20260809 04:56:24.750211 4186 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff11805eb237495690d8d3e9c30e2457" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43283 } }
I20260809 04:56:24.750568 4186 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:24.750666 4186 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff11805eb237495690d8d3e9c30e2457, State: Initialized, Role: FOLLOWER
I20260809 04:56:24.751227 4186 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff11805eb237495690d8d3e9c30e2457" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43283 } }
I20260809 04:56:24.751505 4186 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:24.751674 4186 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:24.751884 4186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:24.754019 4186 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff11805eb237495690d8d3e9c30e2457" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43283 } }
I20260809 04:56:24.754496 4186 leader_election.cc:302] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff11805eb237495690d8d3e9c30e2457; no voters:
I20260809 04:56:24.755151 4186 leader_election.cc:288] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:24.755257 4194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:24.756002 4194 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [term 1 LEADER]: Becoming Leader. State: Replica: ff11805eb237495690d8d3e9c30e2457, State: Running, Role: LEADER
I20260809 04:56:24.756479 4194 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff11805eb237495690d8d3e9c30e2457" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43283 } }
I20260809 04:56:24.757215 4186 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:24.759385 4195 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff11805eb237495690d8d3e9c30e2457" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff11805eb237495690d8d3e9c30e2457" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43283 } } }
I20260809 04:56:24.759794 4195 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:24.759963 4197 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff11805eb237495690d8d3e9c30e2457. Latest consensus state: current_term: 1 leader_uuid: "ff11805eb237495690d8d3e9c30e2457" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff11805eb237495690d8d3e9c30e2457" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43283 } } }
I20260809 04:56:24.760277 4197 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:24.762253 4208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:24.767695 4208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:24.776780 4208 catalog_manager.cc:1347] Generated new cluster ID: fb7af7d88c9248678eeda1447d485ce6
I20260809 04:56:24.776906 4208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:24.793095 4208 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:24.794893 4208 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:24.804417 4208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff11805eb237495690d8d3e9c30e2457: Generated new TSK 0
I20260809 04:56:24.805475 4208 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:56:24.858574 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43283
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:25.197495 4229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:25.197772 4229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:25.197866 4229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:25.205097 4229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:25.205204 4229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:25.205315 4229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:56:25.214804 4229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:25.216099 4229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:25.217394 4229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:25.230829 4236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:25.230787 4237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:25.230741 4239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:25.231606 4229 server_base.cc:1034] running on GCE node
I20260809 04:56:25.232216 4229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:25.233121 4229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:25.234330 4229 hybrid_clock.cc:648] HybridClock initialized: now 1786251385234271 us; error 50 us; skew 500 ppm
I20260809 04:56:25.234632 4229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:25 dist-test-slave-21b9 krb5kdc[4056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251385, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:25.237668 4229 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:56:25.239041 4229 webserver.cc:460] Webserver started at http://127.29.243.193:40003/ using document root <none> and password file <none>
I20260809 04:56:25.239786 4229 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:25.239892 4229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:25.240164 4229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:25.243029 4229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "578625d1b5a940e6a7df02e192ab90b6"
format_stamp: "Formatted at 2026-08-09 04:56:25 on dist-test-slave-21b9"
server_key: "a98cf44f0337575f1ea7a9e998d48afc"
server_key_iv: "4e0a837a9357de9615e11525354a4e1e"
server_key_version: "encryptionkey@0"
I20260809 04:56:25.243834 4229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "578625d1b5a940e6a7df02e192ab90b6"
format_stamp: "Formatted at 2026-08-09 04:56:25 on dist-test-slave-21b9"
server_key: "a98cf44f0337575f1ea7a9e998d48afc"
server_key_iv: "4e0a837a9357de9615e11525354a4e1e"
server_key_version: "encryptionkey@0"
I20260809 04:56:25.248924 4229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:56:25.252532 4251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:25.253870 4229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:56:25.254045 4229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "578625d1b5a940e6a7df02e192ab90b6"
format_stamp: "Formatted at 2026-08-09 04:56:25 on dist-test-slave-21b9"
server_key: "a98cf44f0337575f1ea7a9e998d48afc"
server_key_iv: "4e0a837a9357de9615e11525354a4e1e"
server_key_version: "encryptionkey@0"
I20260809 04:56:25.254245 4229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:25.278327 4229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:25.282650 4229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:25.282934 4229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:25.283985 4229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:25.285363 4229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:25.285449 4229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:25.285573 4229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:25.285627 4229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:25.319373 4229 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:45593
I20260809 04:56:25.319413 4425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:45593 every 8 connection(s)
I20260809 04:56:25.320828 4229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:56:25.331041 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 4229
I20260809 04:56:25.331260 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:56:25.332082 30671 external_mini_cluster.cc:1428] Setting key 83a6de65291d7d75348d83c3b2fea0d6
Aug 09 04:56:25 dist-test-slave-21b9 krb5kdc[4056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251385, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:25.341969 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:25.323476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:53905 (local address 127.29.243.254:43283)
0809 04:56:25.323943 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:25.323949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:25.324921 (+ 972us) server_negotiation.cc:407] Connection header received
0809 04:56:25.326722 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:25.326726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:25.326796 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:25.326891 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:25.328557 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:25.329264 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:25.330894 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:25.331511 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:25.331652 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:25.335416 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:25.335423 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:25.335429 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:25.335474 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:25.338789 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:25.339552 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:25.339559 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:25.339562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:25.339734 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:25.340149 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:25.340155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:25.340158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:25.340515 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:25.340669 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:25.341376 (+ 707us) server_negotiation.cc:299] Negotiation successful
0809 04:56:25.341601 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":217,"threads_started":1}
I20260809 04:56:25.342694 4432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:25.324018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:43283 (local address 127.29.243.193:53905)
0809 04:56:25.324744 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:25.324784 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:25.326469 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:25.327164 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:25.327172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:25.327769 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:25.328386 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:25.328413 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:25.329432 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:25.329437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:25.330729 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:25.330744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:25.331744 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:25.331750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:25.331984 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:25.332688 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:25.332714 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:25.335152 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:25.338929 (+ 3777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:25.338936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:25.338950 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:25.339432 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:25.339835 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:25.339838 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:25.339840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:25.340034 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:25.340625 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:25.340630 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:25.340960 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 04:56:25.341201 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 04:56:25.341593 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":275,"threads_started":1}
I20260809 04:56:25.344310 4426 heartbeater.cc:347] Connected to a master server at 127.29.243.254:43283
I20260809 04:56:25.344620 4426 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:25.345326 4426 heartbeater.cc:511] Master 127.29.243.254:43283 requested a full tablet report, sending...
I20260809 04:56:25.347541 4131 ts_manager.cc:194] Registered new tserver with Master: 578625d1b5a940e6a7df02e192ab90b6 (127.29.243.193:45593)
I20260809 04:56:25.348994 4131 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:53905
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:56:25.384472 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43283
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:25.730052 4438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:25.730422 4438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:25.730581 4438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:25.740803 4438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:25.740918 4438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:25.741037 4438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:56:25.750823 4438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:25.752147 4438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:25.753458 4438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:25.766501 4445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:25.766460 4449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:25.766705 4447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:25.767635 4438 server_base.cc:1034] running on GCE node
I20260809 04:56:25.768262 4438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:25.769220 4438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:25.770440 4438 hybrid_clock.cc:648] HybridClock initialized: now 1786251385770389 us; error 36 us; skew 500 ppm
I20260809 04:56:25.770776 4438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:25 dist-test-slave-21b9 krb5kdc[4056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251385, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:25.774061 4438 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:56:25.775687 4438 webserver.cc:460] Webserver started at http://127.29.243.194:40151/ using document root <none> and password file <none>
I20260809 04:56:25.776418 4438 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:25.776539 4438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:25.776854 4438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:25.779844 4438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "7c3642c1df3b4bc1b672a30f33fe00ef"
format_stamp: "Formatted at 2026-08-09 04:56:25 on dist-test-slave-21b9"
server_key: "9bfaa706fecf4a4863de4e75ab1c2972"
server_key_iv: "6f6657f134f8412894f95ec1542bc66d"
server_key_version: "encryptionkey@0"
I20260809 04:56:25.780730 4438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "7c3642c1df3b4bc1b672a30f33fe00ef"
format_stamp: "Formatted at 2026-08-09 04:56:25 on dist-test-slave-21b9"
server_key: "9bfaa706fecf4a4863de4e75ab1c2972"
server_key_iv: "6f6657f134f8412894f95ec1542bc66d"
server_key_version: "encryptionkey@0"
I20260809 04:56:25.785967 4438 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 04:56:25.789455 4461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:25.790807 4438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:25.790962 4438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "7c3642c1df3b4bc1b672a30f33fe00ef"
format_stamp: "Formatted at 2026-08-09 04:56:25 on dist-test-slave-21b9"
server_key: "9bfaa706fecf4a4863de4e75ab1c2972"
server_key_iv: "6f6657f134f8412894f95ec1542bc66d"
server_key_version: "encryptionkey@0"
I20260809 04:56:25.791152 4438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:25.819448 4438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:25.823671 4438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:25.823947 4438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:25.824925 4438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:25.826295 4438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:25.826385 4438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:25.826504 4438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:25.826557 4438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:25.857121 4438 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:43129
I20260809 04:56:25.857190 4635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:43129 every 8 connection(s)
I20260809 04:56:25.858716 4438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:56:25.866068 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 4438
I20260809 04:56:25.866245 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:56:25.866719 30671 external_mini_cluster.cc:1428] Setting key b1d08d2cd4e5606249f4645f81360358
Aug 09 04:56:25 dist-test-slave-21b9 krb5kdc[4056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251385, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:25.879705 4639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:25.861369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:35525 (local address 127.29.243.254:43283)
0809 04:56:25.861782 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:25.861787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:25.862927 (+ 1140us) server_negotiation.cc:407] Connection header received
0809 04:56:25.864795 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:25.864800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:25.864895 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:25.864995 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:25.866550 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:25.867363 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:25.869260 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:25.869682 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:25.869780 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:25.873784 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:25.873789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:25.873792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:25.873827 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:25.876323 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:25.877132 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:25.877136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:25.877138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:25.877276 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:25.877734 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:25.877739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:25.877742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:25.878085 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:25.878228 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:25.879088 (+ 860us) server_negotiation.cc:299] Negotiation successful
0809 04:56:25.879323 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":189,"threads_started":1}
I20260809 04:56:25.880689 4640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:25.861919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:43283 (local address 127.29.243.194:35525)
0809 04:56:25.862757 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:25.862807 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:25.864529 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:25.865235 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:25.865243 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:25.865774 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:25.866372 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:25.866401 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:25.867551 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:25.867556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:25.869110 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:25.869126 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:25.869910 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:25.869917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:25.870192 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:25.870889 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:25.870918 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:25.873552 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:25.876474 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:25.876483 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:25.876500 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:25.877031 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:25.877370 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:25.877375 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:25.877378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:25.877622 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:25.878200 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:25.878205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:25.878641 (+ 436us) client_negotiation.cc:769] Sending connection context
0809 04:56:25.878972 (+ 331us) client_negotiation.cc:240] Negotiation successful
0809 04:56:25.879464 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":266,"threads_started":1}
I20260809 04:56:25.882303 4637 heartbeater.cc:347] Connected to a master server at 127.29.243.254:43283
I20260809 04:56:25.882640 4637 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:25.883599 4637 heartbeater.cc:511] Master 127.29.243.254:43283 requested a full tablet report, sending...
I20260809 04:56:25.885382 4131 ts_manager.cc:194] Registered new tserver with Master: 7c3642c1df3b4bc1b672a30f33fe00ef (127.29.243.194:43129)
I20260809 04:56:25.886274 4131 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:35525
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:56:25.918670 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43283
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:34061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:26.253772 4649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:26.254065 4649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:26.254200 4649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:26.261795 4649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:26.261915 4649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:26.262070 4649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:56:26.272010 4649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:34061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:26.273348 4649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:26.274668 4649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:26.287864 4658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:26.287894 4660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:26.287928 4664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:26.288722 4649 server_base.cc:1034] running on GCE node
I20260809 04:56:26.289353 4649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:26.290311 4649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:26.291630 4649 hybrid_clock.cc:648] HybridClock initialized: now 1786251386291563 us; error 45 us; skew 500 ppm
I20260809 04:56:26.291972 4649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:26 dist-test-slave-21b9 krb5kdc[4056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251386, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:26.295104 4649 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:56:26.296626 4649 webserver.cc:460] Webserver started at http://127.29.243.195:39927/ using document root <none> and password file <none>
I20260809 04:56:26.297343 4649 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:26.297451 4649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:26.297762 4649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:26.300712 4649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "9031af3c91bd4712a29b09107b46c2ab"
format_stamp: "Formatted at 2026-08-09 04:56:26 on dist-test-slave-21b9"
server_key: "dd27182d2f6aafd1c8b62c0389f969ae"
server_key_iv: "8a8abdb373b42e9a05862b8a6283f686"
server_key_version: "encryptionkey@0"
I20260809 04:56:26.301469 4649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "9031af3c91bd4712a29b09107b46c2ab"
format_stamp: "Formatted at 2026-08-09 04:56:26 on dist-test-slave-21b9"
server_key: "dd27182d2f6aafd1c8b62c0389f969ae"
server_key_iv: "8a8abdb373b42e9a05862b8a6283f686"
server_key_version: "encryptionkey@0"
I20260809 04:56:26.306551 4649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 04:56:26.310101 4677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:26.311471 4649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:26.311642 4649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "9031af3c91bd4712a29b09107b46c2ab"
format_stamp: "Formatted at 2026-08-09 04:56:26 on dist-test-slave-21b9"
server_key: "dd27182d2f6aafd1c8b62c0389f969ae"
server_key_iv: "8a8abdb373b42e9a05862b8a6283f686"
server_key_version: "encryptionkey@0"
I20260809 04:56:26.311823 4649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:26.325800 4649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:26.329950 4649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:26.330217 4649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:26.331158 4649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:26.332464 4649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:26.332538 4649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:26.332649 4649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:26.332705 4649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:26.352046 4426 heartbeater.cc:503] Master 127.29.243.254:43283 was elected leader, sending a full tablet report...
I20260809 04:56:26.363656 4649 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:38837
I20260809 04:56:26.363693 4865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:38837 every 8 connection(s)
I20260809 04:56:26.364948 4649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:56:26.371243 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 4649
I20260809 04:56:26.371465 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:56:26.372036 30671 external_mini_cluster.cc:1428] Setting key f70d3207054085fbe29c0629a3d34384
Aug 09 04:56:26 dist-test-slave-21b9 krb5kdc[4056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251386, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:26.385377 4639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:26.367523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:59831 (local address 127.29.243.254:43283)
0809 04:56:26.367700 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:26.367705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:26.368815 (+ 1110us) server_negotiation.cc:407] Connection header received
0809 04:56:26.370599 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:26.370604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:26.370689 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:26.370784 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:26.372828 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.373679 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.375631 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.376031 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.376126 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:26.379466 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:26.379471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:26.379475 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:26.379509 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:26.382308 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.383014 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.383018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.383020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.383154 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.383617 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.383621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.383623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.383903 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:26.384053 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:26.384772 (+ 719us) server_negotiation.cc:299] Negotiation successful
0809 04:56:26.385009 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:56:26.386471 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:26.367942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:43283 (local address 127.29.243.195:59831)
0809 04:56:26.368647 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:26.368691 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:26.370318 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:26.371106 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:26.371117 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:26.371841 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:26.372646 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:26.372667 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.373843 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.373848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:26.375497 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:26.375511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.376211 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.376215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:26.376413 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:26.377082 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:26.377105 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:26.379225 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:26.382451 (+ 3226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:26.382458 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:26.382471 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:26.382886 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:26.383260 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:26.383263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:26.383291 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:26.383499 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:26.384042 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:26.384051 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:26.384384 (+ 333us) client_negotiation.cc:769] Sending connection context
0809 04:56:26.384624 (+ 240us) client_negotiation.cc:240] Negotiation successful
0809 04:56:26.385178 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":262,"threads_started":1}
I20260809 04:56:26.387799 4866 heartbeater.cc:347] Connected to a master server at 127.29.243.254:43283
I20260809 04:56:26.388085 4866 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:26.388808 4866 heartbeater.cc:511] Master 127.29.243.254:43283 requested a full tablet report, sending...
I20260809 04:56:26.390244 4128 ts_manager.cc:194] Registered new tserver with Master: 9031af3c91bd4712a29b09107b46c2ab (127.29.243.195:38837)
I20260809 04:56:26.391131 4128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:59831
I20260809 04:56:26.400002 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:56:26 dist-test-slave-21b9 krb5kdc[4056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:56:26 dist-test-slave-21b9 krb5kdc[4056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:56:26.452565 4882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:26.438342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58458 (local address 127.29.243.193:45593)
0809 04:56:26.438999 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:26.439007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:26.439035 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:56:26.439109 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:26.439115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:26.439366 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:26.439512 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:26.440853 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.441875 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.443024 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.443644 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.443741 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:26.447455 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:26.447468 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:26.447484 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:26.447522 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:26.450160 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.450749 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.450756 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.450758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.450874 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.451173 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.451176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.451178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.451594 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:26.451825 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:26.452066 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 04:56:26.452259 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":335,"threads_started":1}
Aug 09 04:56:26 dist-test-slave-21b9 krb5kdc[4056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:26.470260 4639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:26.457129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57952 (local address 127.29.243.254:43283)
0809 04:56:26.457336 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:26.457340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:26.457429 (+ 89us) server_negotiation.cc:407] Connection header received
0809 04:56:26.457587 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:26.457590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:26.457669 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:26.457752 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:26.459150 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.459951 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.461093 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.461537 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.461624 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:26.465468 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:26.465473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:26.465477 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:26.465510 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:26.467880 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.468542 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.468546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.468548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.468686 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.468967 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.468970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.468972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.469274 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:26.469472 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:26.469768 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 04:56:26.469937 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 04:56:26.470888 4128 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57952
I20260809 04:56:26.471923 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 4229
I20260809 04:56:26.483706 4882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:26.472737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58460 (local address 127.29.243.193:45593)
0809 04:56:26.472932 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:26.472937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:26.473201 (+ 264us) server_negotiation.cc:407] Connection header received
0809 04:56:26.473333 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:26.473336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:26.473429 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:26.473502 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:26.474589 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.475240 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.476379 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.476771 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.476868 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:26.478862 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:26.478866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:26.478870 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:26.478901 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:26.481550 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.482131 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.482136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.482139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.482282 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.482567 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.482570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.482572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.482813 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:26.482989 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:26.483205 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 04:56:26.483406 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 04:56:26.484268 4420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:26.565621 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 4229
I20260809 04:56:26.588058 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 4438
Aug 09 04:56:26 dist-test-slave-21b9 krb5kdc[4056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:56:26.604465 4891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:26.589030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.29.243.194:43129)
0809 04:56:26.589625 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:26.589632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:26.589659 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:56:26.589726 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:26.589733 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:26.589955 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:26.590089 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:26.591329 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.592436 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.593681 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.594275 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.594372 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:26.599168 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:26.599180 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:26.599197 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:26.599242 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:26.601792 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.602480 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.602488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.602490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.602619 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.602964 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.602968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.602970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.603386 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:26.603641 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:26.603875 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 04:56:26.604098 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":268,"threads_started":1}
I20260809 04:56:26.605711 4626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:26.687100 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 4438
I20260809 04:56:26.713574 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 4649
Aug 09 04:56:26 dist-test-slave-21b9 krb5kdc[4056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:56:26.730958 4894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:26.714554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47526 (local address 127.29.243.195:38837)
0809 04:56:26.715332 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:26.715343 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:26.715389 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:56:26.715478 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:26.715487 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:26.715785 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:26.716008 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:26.717271 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.719018 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.720378 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:26.721304 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:26.721450 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:26.725672 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:26.725683 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:26.725698 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:26.725732 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:26.728312 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.728947 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.728955 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.728957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.729117 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:26.729443 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:26.729446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:26.729448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:26.729844 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:26.730102 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:26.730328 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 04:56:26.730561 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":390,"threads_started":1}
I20260809 04:56:26.732416 4860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:26.814867 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 4649
I20260809 04:56:26.840682 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 4075
I20260809 04:56:26.841919 4175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:26.946048 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 4075
2026-08-09T04:56:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4090 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:56:27 dist-test-slave-21b9 krb5kdc[4907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:56:27 dist-test-slave-21b9 krb5kdc[4907](info): set up 2 sockets
Aug 09 04:56:27 dist-test-slave-21b9 krb5kdc[4907](info): commencing operation
krb5kdc: starting...
W20260809 04:56:28.178191 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:56:28 dist-test-slave-21b9 krb5kdc[4907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:56:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:56:28.406936 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41159
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:28.741453 4927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:28.741770 4927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:28.741920 4927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:28.749203 4927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:28.749320 4927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:28.749399 4927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:28.749477 4927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:28.749545 4927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:28.759408 4927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:28.760766 4927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:28.762117 4927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:28.773242 4932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:28.773201 4935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:28.773275 4933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:28.774168 4927 server_base.cc:1034] running on GCE node
I20260809 04:56:28.774891 4927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:28.776306 4927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:28.777511 4927 hybrid_clock.cc:648] HybridClock initialized: now 1786251388777465 us; error 40 us; skew 500 ppm
I20260809 04:56:28.777840 4927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:28 dist-test-slave-21b9 krb5kdc[4907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251388, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:28.780910 4927 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:56:28.782086 4927 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:28.782193 4927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:28.782514 4927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:28.785492 4927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "faeb387fa8324dd0885d8d5d28465db8"
format_stamp: "Formatted at 2026-08-09 04:56:28 on dist-test-slave-21b9"
server_key: "e4fddee7a20ef848c5c9039e4d9c78d8"
server_key_iv: "f6504968952110e218d942b870e2388c"
server_key_version: "encryptionkey@0"
I20260809 04:56:28.786293 4927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "faeb387fa8324dd0885d8d5d28465db8"
format_stamp: "Formatted at 2026-08-09 04:56:28 on dist-test-slave-21b9"
server_key: "e4fddee7a20ef848c5c9039e4d9c78d8"
server_key_iv: "f6504968952110e218d942b870e2388c"
server_key_version: "encryptionkey@0"
I20260809 04:56:28.791477 4927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260809 04:56:28.795364 4947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:28.796815 4927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:56:28.797001 4927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "faeb387fa8324dd0885d8d5d28465db8"
format_stamp: "Formatted at 2026-08-09 04:56:28 on dist-test-slave-21b9"
server_key: "e4fddee7a20ef848c5c9039e4d9c78d8"
server_key_iv: "f6504968952110e218d942b870e2388c"
server_key_version: "encryptionkey@0"
I20260809 04:56:28.797230 4927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:28.813041 4927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:28.817201 4927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:28.817464 4927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:28.835688 4927 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:41159
I20260809 04:56:28.835711 5029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:41159 every 8 connection(s)
I20260809 04:56:28.836578 4927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:28.839429 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 4927
I20260809 04:56:28.839624 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:28.840371 5030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:28.840374 30671 external_mini_cluster.cc:1428] Setting key ced7f4cd8824d262efe329b467b652f2
I20260809 04:56:28.855142 5030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8: Bootstrap starting.
Aug 09 04:56:28 dist-test-slave-21b9 krb5kdc[4907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:28.859035 5030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:28.860417 5030 log.cc:824] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:28.863914 5030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8: No bootstrap required, opened a new log
I20260809 04:56:28.865286 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:28.842429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45820 (local address 127.29.243.254:41159)
0809 04:56:28.843388 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:28.843407 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:28.843469 (+ 62us) server_negotiation.cc:407] Connection header received
0809 04:56:28.845679 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:28.845710 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:28.846198 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:28.846646 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:28.848029 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:28.850026 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:28.851507 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:28.852589 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:28.852790 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:28.856680 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:28.856696 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:28.856716 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:28.856773 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:28.860658 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:28.861488 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:28.861498 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:28.861504 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:28.861678 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:28.862045 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:28.862050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:28.862053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:28.862805 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:28.863106 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:28.863550 (+ 444us) server_negotiation.cc:299] Negotiation successful
0809 04:56:28.863980 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":411,"threads_started":1}
I20260809 04:56:28.872470 5030 raft_consensus.cc:348] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeb387fa8324dd0885d8d5d28465db8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41159 } }
I20260809 04:56:28.872818 5030 raft_consensus.cc:374] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:28.872896 5030 raft_consensus.cc:729] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faeb387fa8324dd0885d8d5d28465db8, State: Initialized, Role: FOLLOWER
I20260809 04:56:28.873409 5030 consensus_queue.cc:260] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeb387fa8324dd0885d8d5d28465db8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41159 } }
I20260809 04:56:28.873601 5030 raft_consensus.cc:388] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:28.873698 5030 raft_consensus.cc:482] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:28.873817 5030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:28.875964 5030 raft_consensus.cc:504] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeb387fa8324dd0885d8d5d28465db8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41159 } }
I20260809 04:56:28.876406 5030 leader_election.cc:302] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faeb387fa8324dd0885d8d5d28465db8; no voters:
I20260809 04:56:28.876993 5030 leader_election.cc:288] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:28.877157 5037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:28.878221 5037 raft_consensus.cc:686] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [term 1 LEADER]: Becoming Leader. State: Replica: faeb387fa8324dd0885d8d5d28465db8, State: Running, Role: LEADER
I20260809 04:56:28.878698 5037 consensus_queue.cc:237] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeb387fa8324dd0885d8d5d28465db8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41159 } }
I20260809 04:56:28.879168 5030 sys_catalog.cc:564] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:28.881565 5040 sys_catalog.cc:455] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faeb387fa8324dd0885d8d5d28465db8. Latest consensus state: current_term: 1 leader_uuid: "faeb387fa8324dd0885d8d5d28465db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeb387fa8324dd0885d8d5d28465db8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41159 } } }
I20260809 04:56:28.881590 5038 sys_catalog.cc:455] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faeb387fa8324dd0885d8d5d28465db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeb387fa8324dd0885d8d5d28465db8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41159 } } }
I20260809 04:56:28.881990 5040 sys_catalog.cc:458] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:28.882061 5038 sys_catalog.cc:458] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:28.885871 5048 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:28.891855 5048 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:28.900074 5048 catalog_manager.cc:1347] Generated new cluster ID: 4e2ea6005f5b4003a377e8f3ae544932
I20260809 04:56:28.900187 5048 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:28.910924 5048 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:28.912248 5048 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:28.919600 5048 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8: Generated new TSK 0
I20260809 04:56:28.920362 5048 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:56:28.973098 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41159
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:29.310665 5078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:29.310937 5078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:29.311064 5078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:29.318249 5078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:29.318358 5078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:29.318511 5078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:56:29.328331 5078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:29.329684 5078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:29.331007 5078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:29.343668 5084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:29.343617 5094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:29.343701 5087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:29.344445 5078 server_base.cc:1034] running on GCE node
I20260809 04:56:29.345069 5078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:29.345980 5078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:29.347188 5078 hybrid_clock.cc:648] HybridClock initialized: now 1786251389347134 us; error 42 us; skew 500 ppm
I20260809 04:56:29.347592 5078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:29 dist-test-slave-21b9 krb5kdc[4907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251389, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:29.350759 5078 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:56:29.351943 5078 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:29.352056 5078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:29.352365 5078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:29.355353 5078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "6d2ef1a6881a4a52abe752f4579baf1e"
format_stamp: "Formatted at 2026-08-09 04:56:29 on dist-test-slave-21b9"
server_key: "27875b476ab332e8d755938d53f2c9d6"
server_key_iv: "fd0baa6e6474434d9ecd941bcf2734c4"
server_key_version: "encryptionkey@0"
I20260809 04:56:29.356171 5078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "6d2ef1a6881a4a52abe752f4579baf1e"
format_stamp: "Formatted at 2026-08-09 04:56:29 on dist-test-slave-21b9"
server_key: "27875b476ab332e8d755938d53f2c9d6"
server_key_iv: "fd0baa6e6474434d9ecd941bcf2734c4"
server_key_version: "encryptionkey@0"
I20260809 04:56:29.361156 5078 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:56:29.364737 5102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:29.366060 5078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:56:29.366219 5078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "6d2ef1a6881a4a52abe752f4579baf1e"
format_stamp: "Formatted at 2026-08-09 04:56:29 on dist-test-slave-21b9"
server_key: "27875b476ab332e8d755938d53f2c9d6"
server_key_iv: "fd0baa6e6474434d9ecd941bcf2734c4"
server_key_version: "encryptionkey@0"
I20260809 04:56:29.366410 5078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:29.381337 5078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:29.385480 5078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:29.385744 5078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:29.386718 5078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:29.388140 5078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:29.388229 5078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:29.388334 5078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:29.388406 5078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:29.417925 5078 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:38137
I20260809 04:56:29.417973 5290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:38137 every 8 connection(s)
I20260809 04:56:29.418777 5078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:56:29.425047 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 5078
I20260809 04:56:29.425238 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:56:29.425873 30671 external_mini_cluster.cc:1428] Setting key 0dad716d409918c2fd7fb9a779d8e3fc
Aug 09 04:56:29 dist-test-slave-21b9 krb5kdc[4907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251389, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:29.438586 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:29.421239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:57313 (local address 127.29.243.254:41159)
0809 04:56:29.421752 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:29.421757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:29.422637 (+ 880us) server_negotiation.cc:407] Connection header received
0809 04:56:29.424460 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:29.424475 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:29.424563 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:29.424681 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:29.426255 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:29.427087 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:29.428797 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:29.429209 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:29.429318 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:29.432942 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:29.432947 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:29.432950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:29.432985 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:29.435498 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:29.436311 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:29.436314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:29.436317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:29.436423 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:29.436841 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:29.436845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:29.436848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:29.437197 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:29.437343 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:29.438003 (+ 660us) server_negotiation.cc:299] Negotiation successful
0809 04:56:29.438207 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":257,"threads_started":1}
I20260809 04:56:29.439389 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:29.421784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41159 (local address 127.29.243.193:57313)
0809 04:56:29.422466 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:29.422520 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:29.424192 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:29.424908 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:29.424916 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:29.425503 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:29.426093 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:29.426111 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:29.427249 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:29.427254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:29.428649 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:29.428662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:29.429435 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:29.429443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:29.429654 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:29.430288 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:29.430318 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:29.432725 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:29.435640 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:29.435649 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:29.435665 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:29.436212 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:29.436555 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:29.436558 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:29.436561 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:29.436737 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:29.437300 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:29.437305 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:29.437640 (+ 335us) client_negotiation.cc:769] Sending connection context
0809 04:56:29.437867 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 04:56:29.438261 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":266,"threads_started":1}
I20260809 04:56:29.440999 5292 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41159
I20260809 04:56:29.441334 5292 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:29.442184 5292 heartbeater.cc:511] Master 127.29.243.254:41159 requested a full tablet report, sending...
I20260809 04:56:29.444389 4971 ts_manager.cc:194] Registered new tserver with Master: 6d2ef1a6881a4a52abe752f4579baf1e (127.29.243.193:38137)
I20260809 04:56:29.445930 4971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:57313
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:56:29.478469 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41159
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:29.812958 5301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:29.813248 5301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:29.813401 5301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:29.820493 5301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:29.820612 5301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:29.820765 5301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:56:29.830534 5301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:29.831909 5301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:29.833218 5301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:29.847756 5317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:29.847779 5315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:29.848235 5301 server_base.cc:1034] running on GCE node
W20260809 04:56:29.848412 5314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:29.849093 5301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:29.850029 5301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:29.851287 5301 hybrid_clock.cc:648] HybridClock initialized: now 1786251389851199 us; error 56 us; skew 500 ppm
I20260809 04:56:29.851619 5301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:29 dist-test-slave-21b9 krb5kdc[4907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251389, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:29.854655 5301 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:56:29.855856 5301 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:29.855962 5301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:29.856280 5301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:29.859407 5301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "6663d0a0e6a5401692e72ebc3bddab02"
format_stamp: "Formatted at 2026-08-09 04:56:29 on dist-test-slave-21b9"
server_key: "b22149a5dff0beb6d9a2a48169d82cdd"
server_key_iv: "13755e01a4c10c49697bb4f0f0557a3c"
server_key_version: "encryptionkey@0"
I20260809 04:56:29.860195 5301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "6663d0a0e6a5401692e72ebc3bddab02"
format_stamp: "Formatted at 2026-08-09 04:56:29 on dist-test-slave-21b9"
server_key: "b22149a5dff0beb6d9a2a48169d82cdd"
server_key_iv: "13755e01a4c10c49697bb4f0f0557a3c"
server_key_version: "encryptionkey@0"
I20260809 04:56:29.865319 5301 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 04:56:29.868935 5325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:29.870479 5301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:56:29.870647 5301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "6663d0a0e6a5401692e72ebc3bddab02"
format_stamp: "Formatted at 2026-08-09 04:56:29 on dist-test-slave-21b9"
server_key: "b22149a5dff0beb6d9a2a48169d82cdd"
server_key_iv: "13755e01a4c10c49697bb4f0f0557a3c"
server_key_version: "encryptionkey@0"
I20260809 04:56:29.870867 5301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:29.892652 5301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:29.896517 5301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:29.896763 5301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:29.897745 5301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:29.899178 5301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:29.899295 5301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:29.899456 5301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:29.899547 5301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:29.931094 5301 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:40431
I20260809 04:56:29.931149 5513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:40431 every 8 connection(s)
I20260809 04:56:29.932026 5301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:56:29.940462 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 5301
I20260809 04:56:29.940618 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:56:29.941093 30671 external_mini_cluster.cc:1428] Setting key 980b638ff5da949cf3888eab43f206f7
Aug 09 04:56:29 dist-test-slave-21b9 krb5kdc[4907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251389, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:29.951918 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:29.934508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:51983 (local address 127.29.243.254:41159)
0809 04:56:29.934687 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:29.934691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:29.935889 (+ 1198us) server_negotiation.cc:407] Connection header received
0809 04:56:29.937696 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:29.937702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:29.937789 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:29.937884 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:29.939414 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:29.940141 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:29.941900 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:29.942529 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:29.942647 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:29.946381 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:29.946388 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:29.946393 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:29.946427 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:29.948856 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:29.949620 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:29.949625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:29.949627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:29.949762 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:29.950173 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:29.950177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:29.950179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:29.950463 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:29.950599 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:29.951310 (+ 711us) server_negotiation.cc:299] Negotiation successful
0809 04:56:29.951558 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:56:29.952867 5518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:29.934956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41159 (local address 127.29.243.194:51983)
0809 04:56:29.935707 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:29.935753 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:29.937433 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:29.938126 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:29.938135 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:29.938659 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:29.939216 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:29.939232 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:29.940296 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:29.940300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:29.941713 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:29.941729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:29.942790 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:29.942799 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:29.943096 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:29.943853 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:29.943876 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:29.946157 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:29.948995 (+ 2838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:29.949004 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:29.949021 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:29.949496 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:29.949867 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:29.949872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:29.949876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:29.950051 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:29.950589 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:29.950597 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:29.950915 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 04:56:29.951156 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 04:56:29.951703 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":333,"threads_started":1}
I20260809 04:56:29.954314 5514 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41159
I20260809 04:56:29.954613 5514 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:29.955260 5514 heartbeater.cc:511] Master 127.29.243.254:41159 requested a full tablet report, sending...
I20260809 04:56:29.956725 4971 ts_manager.cc:194] Registered new tserver with Master: 6663d0a0e6a5401692e72ebc3bddab02 (127.29.243.194:40431)
I20260809 04:56:29.957501 4971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:51983
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:56:29.990576 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41159
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:30.326807 5521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:30.327086 5521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:30.327183 5521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:30.334451 5521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:30.334573 5521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:30.334703 5521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:56:30.344678 5521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:30.346082 5521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:30.347540 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:30.360471 5527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:30.360556 5528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:30.360642 5530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:30.361593 5521 server_base.cc:1034] running on GCE node
I20260809 04:56:30.362200 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:30.363163 5521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:30.364406 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786251390364338 us; error 44 us; skew 500 ppm
I20260809 04:56:30.364754 5521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:30 dist-test-slave-21b9 krb5kdc[4907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251390, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:30.367934 5521 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:56:30.369138 5521 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:30.369256 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:30.369570 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:30.372615 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "29900a8d8e9b468a893814f0b57a5f99"
format_stamp: "Formatted at 2026-08-09 04:56:30 on dist-test-slave-21b9"
server_key: "60ad671160a4e78d4d6888d6e1d05c4b"
server_key_iv: "bd673a5f17d3945ff0d39aa036364b06"
server_key_version: "encryptionkey@0"
I20260809 04:56:30.373401 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "29900a8d8e9b468a893814f0b57a5f99"
format_stamp: "Formatted at 2026-08-09 04:56:30 on dist-test-slave-21b9"
server_key: "60ad671160a4e78d4d6888d6e1d05c4b"
server_key_iv: "bd673a5f17d3945ff0d39aa036364b06"
server_key_version: "encryptionkey@0"
I20260809 04:56:30.378486 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 04:56:30.382267 5539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:30.383875 5521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:30.384032 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "29900a8d8e9b468a893814f0b57a5f99"
format_stamp: "Formatted at 2026-08-09 04:56:30 on dist-test-slave-21b9"
server_key: "60ad671160a4e78d4d6888d6e1d05c4b"
server_key_iv: "bd673a5f17d3945ff0d39aa036364b06"
server_key_version: "encryptionkey@0"
I20260809 04:56:30.384240 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:30.399329 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:30.403513 5521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:30.403767 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:30.404742 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:30.406077 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:30.406164 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:30.406270 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:30.406347 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:30.438063 5521 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:40609
I20260809 04:56:30.438117 5710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:40609 every 8 connection(s)
I20260809 04:56:30.438931 5521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:56:30.442545 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 5521
I20260809 04:56:30.442745 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:56:30.443336 30671 external_mini_cluster.cc:1428] Setting key 4a874d3b4a8ecda76742a2fccbfa7661
I20260809 04:56:30.449471 5292 heartbeater.cc:503] Master 127.29.243.254:41159 was elected leader, sending a full tablet report...
Aug 09 04:56:30 dist-test-slave-21b9 krb5kdc[4907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251390, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:30.459249 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.441744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:38719 (local address 127.29.243.254:41159)
0809 04:56:30.441963 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:30.441969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:30.443203 (+ 1234us) server_negotiation.cc:407] Connection header received
0809 04:56:30.445180 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:30.445186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:30.445276 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:30.445380 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:30.447118 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.447931 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.449492 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.449966 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.450095 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.453653 (+ 3558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:30.453658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:30.453661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:30.453692 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:30.456163 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:30.456865 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:30.456870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:30.456874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:30.457035 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:30.457474 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:30.457478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:30.457481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:30.457833 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:30.457977 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:30.458683 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 04:56:30.458907 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:56:30.460264 5714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.442218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41159 (local address 127.29.243.195:38719)
0809 04:56:30.443043 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:30.443088 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:30.444813 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:30.445815 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:30.445827 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:30.446351 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:30.446925 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.446943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.448068 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.448072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.449350 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.449363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.450199 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.450209 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.450488 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.451232 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:30.451260 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:30.453451 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:30.456294 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:30.456300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:30.456312 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:30.456759 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:30.457135 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:30.457140 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:30.457143 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:30.457360 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:30.457965 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:30.457973 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:30.458325 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 04:56:30.458568 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 04:56:30.459107 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":348,"threads_started":1}
I20260809 04:56:30.461633 5711 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41159
I20260809 04:56:30.461927 5711 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:30.462592 5711 heartbeater.cc:511] Master 127.29.243.254:41159 requested a full tablet report, sending...
I20260809 04:56:30.464124 4971 ts_manager.cc:194] Registered new tserver with Master: 29900a8d8e9b468a893814f0b57a5f99 (127.29.243.195:40609)
I20260809 04:56:30.464879 4971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:38719
I20260809 04:56:30.470794 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:56:30.487056 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.475331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45830 (local address 127.29.243.254:41159)
0809 04:56:30.475539 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:30.475547 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:30.475981 (+ 434us) server_negotiation.cc:407] Connection header received
0809 04:56:30.476122 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:30.476125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:30.476188 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:30.476265 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:30.477655 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.478366 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.479625 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.480104 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.480197 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.482291 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:30.482296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:30.482300 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:30.482332 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:30.484874 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:30.485464 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:30.485467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:30.485469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:30.485595 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:30.485864 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:30.485866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:30.485868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:30.486190 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:30.486381 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:30.486605 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 04:56:30.486759 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:56:30.491155 4971 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:30.497946 4971 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:30.541069 5743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.511492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41690 (local address 127.29.243.193:38137)
0809 04:56:30.522916 (+ 11424us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:30.522927 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:30.532215 (+ 9288us) server_negotiation.cc:407] Connection header received
0809 04:56:30.532332 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:30.532340 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:30.532578 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:30.532754 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:30.533946 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.535805 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.538357 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.539760 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.539953 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.540489 (+ 536us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:30.540565 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 04:56:30.540730 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11191,"thread_start_us":10767,"threads_started":1}
I20260809 04:56:30.541121 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.510610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.29.243.195:40609)
0809 04:56:30.532412 (+ 21802us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:30.532419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:30.532449 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:56:30.532539 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:30.532546 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:30.532733 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:30.533150 (+ 417us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:30.533918 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.535354 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.538178 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.539674 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.539893 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.540075 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:30.540638 (+ 563us) server_negotiation.cc:299] Negotiation successful
0809 04:56:30.540782 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21631,"thread_start_us":21456,"threads_started":1}
I20260809 04:56:30.541142 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.511402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52398 (local address 127.29.243.194:40431)
0809 04:56:30.525078 (+ 13676us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:30.525087 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:30.531256 (+ 6169us) server_negotiation.cc:407] Connection header received
0809 04:56:30.531431 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:30.531440 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:30.531741 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:30.531954 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:30.534113 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.535729 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.537874 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.539385 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.539590 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.539794 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:30.540637 (+ 843us) server_negotiation.cc:299] Negotiation successful
0809 04:56:30.540830 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13511,"thread_start_us":13418,"threads_started":1}
I20260809 04:56:30.541185 5735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.510167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:40609 (local address 127.0.0.1:55242)
0809 04:56:30.525576 (+ 15409us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:30.525634 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:30.525851 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:30.532958 (+ 7107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:30.532963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:30.533031 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:30.533663 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.533679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.535525 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.535530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.537996 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.538011 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.539876 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.539888 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.540189 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.540274 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 04:56:30.540440 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 04:56:30.540677 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15106,"mutex_wait_us":82,"spinlock_wait_cycles":3968,"thread_start_us":15904,"threads_started":1}
I20260809 04:56:30.541890 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.510733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:40431 (local address 127.0.0.1:52398)
0809 04:56:30.531011 (+ 20278us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:30.531035 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:30.531179 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:30.532452 (+ 1273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:30.532460 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:30.532493 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:30.533892 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.533904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.535939 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.535945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.537692 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.537706 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.539963 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.539970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.540351 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.540390 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 04:56:30.540492 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 04:56:30.541456 (+ 964us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20133,"mutex_wait_us":967,"thread_start_us":20696,"threads_started":1}
I20260809 04:56:30.542196 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.510831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38137 (local address 127.0.0.1:41690)
0809 04:56:30.531685 (+ 20854us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:30.531706 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:30.531822 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:30.532925 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:30.532931 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:30.532956 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:30.533677 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.533688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.535955 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.535960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.538218 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.538230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.540186 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.540191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.540424 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.540456 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:56:30.540527 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 04:56:30.541834 (+ 1307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20724,"mutex_wait_us":97,"thread_start_us":19907,"threads_started":1}
I20260809 04:56:30.545672 5176 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe0d61d34ac4f13892aa6b425b02f40 (DEFAULT_TABLE table=test-table [id=b79574ece8f94d9f81e0a4e982592bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:30.545919 5612 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe0d61d34ac4f13892aa6b425b02f40 (DEFAULT_TABLE table=test-table [id=b79574ece8f94d9f81e0a4e982592bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:30.545919 5413 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe0d61d34ac4f13892aa6b425b02f40 (DEFAULT_TABLE table=test-table [id=b79574ece8f94d9f81e0a4e982592bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:30.547062 5176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe0d61d34ac4f13892aa6b425b02f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:30.547425 5612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe0d61d34ac4f13892aa6b425b02f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:30.547423 5413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe0d61d34ac4f13892aa6b425b02f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:30.558923 5747 tablet_bootstrap.cc:492] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: Bootstrap starting.
I20260809 04:56:30.562242 5747 tablet_bootstrap.cc:654] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:30.562614 5749 tablet_bootstrap.cc:492] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: Bootstrap starting.
I20260809 04:56:30.563755 5750 tablet_bootstrap.cc:492] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: Bootstrap starting.
I20260809 04:56:30.565038 5747 log.cc:824] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:30.566982 5749 tablet_bootstrap.cc:654] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:30.567175 5750 tablet_bootstrap.cc:654] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:30.568156 5749 log.cc:824] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:30.568409 5750 log.cc:824] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:30.568513 5747 tablet_bootstrap.cc:492] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: No bootstrap required, opened a new log
I20260809 04:56:30.568778 5747 ts_tablet_manager.cc:1397] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260809 04:56:30.578101 5750 tablet_bootstrap.cc:492] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: No bootstrap required, opened a new log
I20260809 04:56:30.578094 5749 tablet_bootstrap.cc:492] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: No bootstrap required, opened a new log
I20260809 04:56:30.578419 5749 ts_tablet_manager.cc:1397] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260809 04:56:30.578419 5750 ts_tablet_manager.cc:1397] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.007s
I20260809 04:56:30.578940 5747 raft_consensus.cc:348] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29900a8d8e9b468a893814f0b57a5f99" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 40609 } } peers { permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 } } peers { permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 } }
I20260809 04:56:30.579348 5747 raft_consensus.cc:374] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:30.579466 5747 raft_consensus.cc:729] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d2ef1a6881a4a52abe752f4579baf1e, State: Initialized, Role: FOLLOWER
I20260809 04:56:30.580112 5747 consensus_queue.cc:260] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29900a8d8e9b468a893814f0b57a5f99" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 40609 } } peers { permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 } } peers { permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 } }
I20260809 04:56:30.581924 5747 ts_tablet_manager.cc:1428] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260809 04:56:30.589618 5750 raft_consensus.cc:348] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29900a8d8e9b468a893814f0b57a5f99" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 40609 } } peers { permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 } } peers { permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 } }
I20260809 04:56:30.589534 5749 raft_consensus.cc:348] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29900a8d8e9b468a893814f0b57a5f99" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 40609 } } peers { permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 } } peers { permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 } }
I20260809 04:56:30.590045 5750 raft_consensus.cc:374] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:30.590049 5749 raft_consensus.cc:374] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:30.590162 5749 raft_consensus.cc:729] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6663d0a0e6a5401692e72ebc3bddab02, State: Initialized, Role: FOLLOWER
I20260809 04:56:30.590162 5750 raft_consensus.cc:729] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29900a8d8e9b468a893814f0b57a5f99, State: Initialized, Role: FOLLOWER
I20260809 04:56:30.590839 5750 consensus_queue.cc:260] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29900a8d8e9b468a893814f0b57a5f99" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 40609 } } peers { permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 } } peers { permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 } }
I20260809 04:56:30.590850 5749 consensus_queue.cc:260] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29900a8d8e9b468a893814f0b57a5f99" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 40609 } } peers { permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 } } peers { permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 } }
I20260809 04:56:30.592000 5711 heartbeater.cc:503] Master 127.29.243.254:41159 was elected leader, sending a full tablet report...
I20260809 04:56:30.592478 5514 heartbeater.cc:503] Master 127.29.243.254:41159 was elected leader, sending a full tablet report...
I20260809 04:56:30.592870 5750 ts_tablet_manager.cc:1428] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260809 04:56:30.593129 5749 ts_tablet_manager.cc:1428] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260809 04:56:30.671126 5293 tablet.cc:2369] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:56:30.683956 5516 tablet.cc:2369] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:56:30.691423 5712 tablet.cc:2369] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:56:30.765553 5757 raft_consensus.cc:482] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:30.765931 5757 raft_consensus.cc:504] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29900a8d8e9b468a893814f0b57a5f99" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 40609 } } peers { permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 } } peers { permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 } }
I20260809 04:56:30.766415 5758 raft_consensus.cc:482] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:30.766705 5758 raft_consensus.cc:504] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29900a8d8e9b468a893814f0b57a5f99" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 40609 } } peers { permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 } } peers { permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 } }
I20260809 04:56:30.768332 5757 leader_election.cc:288] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29900a8d8e9b468a893814f0b57a5f99 (127.29.243.195:40609), 6d2ef1a6881a4a52abe752f4579baf1e (127.29.243.193:38137)
I20260809 04:56:30.775434 5758 leader_election.cc:288] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6663d0a0e6a5401692e72ebc3bddab02 (127.29.243.194:40431), 6d2ef1a6881a4a52abe752f4579baf1e (127.29.243.193:38137)
I20260809 04:56:30.780165 5714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.768730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:40431 (local address 127.29.243.195:52483)
0809 04:56:30.769207 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:30.769234 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:30.769447 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:30.771453 (+ 2006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:30.771459 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:30.771503 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:30.771719 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.771730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.773366 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.773372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.775510 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.775527 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.779218 (+ 3691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.779225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.779468 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.779524 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 04:56:30.779628 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 04:56:30.779801 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274}
I20260809 04:56:30.781173 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.770406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:52483 (local address 127.29.243.194:40431)
0809 04:56:30.770593 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:30.770600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:30.770623 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:56:30.770702 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:30.770708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:30.770787 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:30.770884 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:30.771897 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.773163 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.776440 (+ 3277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.778881 (+ 2441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.779032 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.779115 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:30.780723 (+ 1608us) server_negotiation.cc:299] Negotiation successful
0809 04:56:30.780892 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 04:56:30.782099 5445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe0d61d34ac4f13892aa6b425b02f40" candidate_uuid: "29900a8d8e9b468a893814f0b57a5f99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6663d0a0e6a5401692e72ebc3bddab02" is_pre_election: true
I20260809 04:56:30.782657 5445 raft_consensus.cc:2454] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29900a8d8e9b468a893814f0b57a5f99 in term 0.
I20260809 04:56:30.783578 5540 leader_election.cc:302] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29900a8d8e9b468a893814f0b57a5f99, 6663d0a0e6a5401692e72ebc3bddab02; no voters:
I20260809 04:56:30.784155 5758 raft_consensus.cc:2781] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:30.785154 5758 raft_consensus.cc:482] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:30.785305 5758 raft_consensus.cc:3037] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:30.785712 5762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.769535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38137 (local address 127.29.243.195:48451)
0809 04:56:30.778583 (+ 9048us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:30.778602 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:30.778717 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:30.779497 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:30.779502 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:30.779527 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:30.779911 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.779923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.781416 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.781421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.783331 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.783346 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.784894 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.784900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.785082 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.785114 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:56:30.785198 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:56:30.785322 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8931,"thread_start_us":8715,"threads_started":1}
I20260809 04:56:30.786253 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.769504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:48451 (local address 127.29.243.193:38137)
0809 04:56:30.778978 (+ 9474us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:30.778984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:30.779005 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:56:30.779067 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:30.779072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:30.779137 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:30.779220 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:30.780107 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.781263 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.783525 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.784690 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.784850 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.784930 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:30.785776 (+ 846us) server_negotiation.cc:299] Negotiation successful
0809 04:56:30.785929 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9328,"thread_start_us":9091,"threads_started":1}
I20260809 04:56:30.787034 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.769052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:44001 (local address 127.29.243.195:40609)
0809 04:56:30.769270 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:30.769276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:30.777828 (+ 8552us) server_negotiation.cc:407] Connection header received
0809 04:56:30.779018 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:30.779023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:30.779087 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:30.779140 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:30.780004 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.781119 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.783809 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.785062 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.785213 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.785296 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:30.786708 (+ 1412us) server_negotiation.cc:299] Negotiation successful
0809 04:56:30.786818 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":85}
I20260809 04:56:30.787631 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.768772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:40609 (local address 127.29.243.194:44001)
0809 04:56:30.777718 (+ 8946us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:30.777740 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:30.778862 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:30.779346 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:30.779350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:30.779376 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:30.779841 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.779854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.781286 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.781290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.783645 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.783659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.785513 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.785519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.785732 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.785764 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:56:30.787198 (+ 1434us) client_negotiation.cc:240] Negotiation successful
0809 04:56:30.787349 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8799,"mutex_wait_us":3,"thread_start_us":15458,"threads_started":1}
I20260809 04:56:30.787792 5214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe0d61d34ac4f13892aa6b425b02f40" candidate_uuid: "29900a8d8e9b468a893814f0b57a5f99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" is_pre_election: true
I20260809 04:56:30.788451 5214 raft_consensus.cc:2454] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29900a8d8e9b468a893814f0b57a5f99 in term 0.
I20260809 04:56:30.788645 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe0d61d34ac4f13892aa6b425b02f40" candidate_uuid: "6663d0a0e6a5401692e72ebc3bddab02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29900a8d8e9b468a893814f0b57a5f99" is_pre_election: true
I20260809 04:56:30.788719 5758 raft_consensus.cc:504] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29900a8d8e9b468a893814f0b57a5f99" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 40609 } } peers { permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 } } peers { permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 } }
I20260809 04:56:30.789350 5644 raft_consensus.cc:2379] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6663d0a0e6a5401692e72ebc3bddab02 in current term 1: Already voted for candidate 29900a8d8e9b468a893814f0b57a5f99 in this term.
I20260809 04:56:30.790064 5743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.768913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:50515 (local address 127.29.243.193:38137)
0809 04:56:30.769757 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:30.769763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:30.781989 (+ 12226us) server_negotiation.cc:407] Connection header received
0809 04:56:30.782253 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:30.782258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:30.782329 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:30.782388 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:30.783190 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.784255 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.786698 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.788967 (+ 2269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.789158 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.789226 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:30.789717 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 04:56:30.789847 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669}
I20260809 04:56:30.790170 5760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.769959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38137 (local address 127.29.243.194:50515)
0809 04:56:30.781883 (+ 11924us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:30.781906 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:30.782087 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:30.782565 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:30.782569 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:30.782593 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:30.783005 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.783018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.784421 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.784426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.786512 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:30.786526 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.789408 (+ 2882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.789414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:30.789562 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.789585 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:56:30.789674 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 04:56:30.789786 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11780,"mutex_wait_us":199,"thread_start_us":7381,"threads_started":1}
I20260809 04:56:30.790592 5445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe0d61d34ac4f13892aa6b425b02f40" candidate_uuid: "29900a8d8e9b468a893814f0b57a5f99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6663d0a0e6a5401692e72ebc3bddab02"
I20260809 04:56:30.790807 5445 raft_consensus.cc:3037] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:30.790807 5214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe0d61d34ac4f13892aa6b425b02f40" candidate_uuid: "6663d0a0e6a5401692e72ebc3bddab02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" is_pre_election: true
I20260809 04:56:30.790982 5758 leader_election.cc:288] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [CANDIDATE]: Term 1 election: Requested vote from peers 6663d0a0e6a5401692e72ebc3bddab02 (127.29.243.194:40431), 6d2ef1a6881a4a52abe752f4579baf1e (127.29.243.193:38137)
I20260809 04:56:30.791011 5214 raft_consensus.cc:2454] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6663d0a0e6a5401692e72ebc3bddab02 in term 0.
I20260809 04:56:30.791368 5213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe0d61d34ac4f13892aa6b425b02f40" candidate_uuid: "29900a8d8e9b468a893814f0b57a5f99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d2ef1a6881a4a52abe752f4579baf1e"
I20260809 04:56:30.791443 5331 leader_election.cc:302] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6663d0a0e6a5401692e72ebc3bddab02, 6d2ef1a6881a4a52abe752f4579baf1e; no voters: 29900a8d8e9b468a893814f0b57a5f99
I20260809 04:56:30.791626 5213 raft_consensus.cc:3037] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:30.793977 5445 raft_consensus.cc:2454] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29900a8d8e9b468a893814f0b57a5f99 in term 1.
I20260809 04:56:30.794269 5757 raft_consensus.cc:2741] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 04:56:30.794361 5540 leader_election.cc:302] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29900a8d8e9b468a893814f0b57a5f99, 6663d0a0e6a5401692e72ebc3bddab02; no voters:
I20260809 04:56:30.794642 5213 raft_consensus.cc:2454] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29900a8d8e9b468a893814f0b57a5f99 in term 1.
I20260809 04:56:30.794657 5758 raft_consensus.cc:2781] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:30.795141 5758 raft_consensus.cc:686] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 1 LEADER]: Becoming Leader. State: Replica: 29900a8d8e9b468a893814f0b57a5f99, State: Running, Role: LEADER
I20260809 04:56:30.795646 5758 consensus_queue.cc:237] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29900a8d8e9b468a893814f0b57a5f99" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 40609 } } peers { permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 } } peers { permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 } }
I20260809 04:56:30.800390 4971 catalog_manager.cc:5495] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29900a8d8e9b468a893814f0b57a5f99 (127.29.243.195). New cstate: current_term: 1 leader_uuid: "29900a8d8e9b468a893814f0b57a5f99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29900a8d8e9b468a893814f0b57a5f99" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 40609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 } health_report { overall_health: UNKNOWN } } }
I20260809 04:56:30.848510 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.842972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55258 (local address 127.29.243.195:40609)
0809 04:56:30.843150 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:30.843157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:30.843331 (+ 174us) server_negotiation.cc:407] Connection header received
0809 04:56:30.843600 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:30.843605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:30.843694 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:30.843830 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:30.844220 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.844989 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.846336 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.846745 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.846857 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.847374 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:30.847896 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:30.848045 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:30.848132 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 04:56:30.848227 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:56:30.857797 5213 raft_consensus.cc:1261] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e [term 1 FOLLOWER]: Refusing update from remote peer 29900a8d8e9b468a893814f0b57a5f99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:30.857791 5445 raft_consensus.cc:1261] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 1 FOLLOWER]: Refusing update from remote peer 29900a8d8e9b468a893814f0b57a5f99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:30.858705 5765 consensus_queue.cc:1035] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d2ef1a6881a4a52abe752f4579baf1e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:30.859099 5758 consensus_queue.cc:1035] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6663d0a0e6a5401692e72ebc3bddab02" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 40431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:30.886626 4970 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:30.886919 4970 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:30.891199 4970 catalog_manager.cc:5782] T 00000000000000000000000000000000 P faeb387fa8324dd0885d8d5d28465db8: Sending DeleteTablet for 3 replicas of tablet 8fe0d61d34ac4f13892aa6b425b02f40
I20260809 04:56:30.892416 5612 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fe0d61d34ac4f13892aa6b425b02f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:30 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:55242
I20260809 04:56:30.892729 5176 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fe0d61d34ac4f13892aa6b425b02f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:30 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:41690
I20260809 04:56:30.893222 5413 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fe0d61d34ac4f13892aa6b425b02f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:30 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:52398
I20260809 04:56:30.895850 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 5078
I20260809 04:56:30.909639 5784 tablet_replica.cc:330] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: stopping tablet replica
I20260809 04:56:30.910140 5784 raft_consensus.cc:2229] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:30.910851 5784 raft_consensus.cc:2258] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:30.913066 5787 tablet_replica.cc:330] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: stopping tablet replica
I20260809 04:56:30.913504 5784 ts_tablet_manager.cc:1905] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:30.913503 5787 raft_consensus.cc:2229] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:30.913983 5787 raft_consensus.cc:2258] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:30.916177 5787 ts_tablet_manager.cc:1905] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:30.923316 5787 ts_tablet_manager.cc:1918] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:30.923517 5787 log.cc:1196] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/8fe0d61d34ac4f13892aa6b425b02f40
I20260809 04:56:30.924081 5787 ts_tablet_manager.cc:1939] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6663d0a0e6a5401692e72ebc3bddab02: Deleting consensus metadata
I20260809 04:56:30.926152 4948 catalog_manager.cc:4841] TS 6663d0a0e6a5401692e72ebc3bddab02 (127.29.243.194:40431): tablet 8fe0d61d34ac4f13892aa6b425b02f40 (table test-table [id=b79574ece8f94d9f81e0a4e982592bf0]) successfully deleted
I20260809 04:56:30.926990 5786 tablet_replica.cc:330] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: stopping tablet replica
I20260809 04:56:30.927510 5786 raft_consensus.cc:2229] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:30.928052 5786 raft_consensus.cc:2258] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 04:56:30 dist-test-slave-21b9 krb5kdc[4907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:56:30.930461 5786 ts_tablet_manager.cc:1905] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:30.938223 5743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:30.897349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.29.243.193:38137)
0809 04:56:30.897553 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:30.897561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:30.912402 (+ 14841us) server_negotiation.cc:407] Connection header received
0809 04:56:30.912648 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:30.912655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:30.912763 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:30.912891 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:30.915371 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.916685 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.919049 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:30.919918 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:30.920038 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:30.930029 (+ 9991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:30.930044 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:30.930062 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:30.930111 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:30.934150 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:30.935148 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:30.935158 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:30.935162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:30.935394 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:30.935940 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:30.935946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:30.935949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:30.936490 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:30.936849 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:30.937505 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 04:56:30.937761 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:56:30.938719 5786 ts_tablet_manager.cc:1918] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:30.938983 5786 log.cc:1196] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/8fe0d61d34ac4f13892aa6b425b02f40
I20260809 04:56:30.939813 5280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:30.945815 5784 ts_tablet_manager.cc:1918] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:30.946012 5784 log.cc:1196] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/8fe0d61d34ac4f13892aa6b425b02f40
I20260809 04:56:30.946602 5784 ts_tablet_manager.cc:1939] T 8fe0d61d34ac4f13892aa6b425b02f40 P 29900a8d8e9b468a893814f0b57a5f99: Deleting consensus metadata
I20260809 04:56:30.948622 4951 catalog_manager.cc:4841] TS 29900a8d8e9b468a893814f0b57a5f99 (127.29.243.195:40609): tablet 8fe0d61d34ac4f13892aa6b425b02f40 (table test-table [id=b79574ece8f94d9f81e0a4e982592bf0]) successfully deleted
I20260809 04:56:31.035792 5786 ts_tablet_manager.cc:1939] T 8fe0d61d34ac4f13892aa6b425b02f40 P 6d2ef1a6881a4a52abe752f4579baf1e: Deleting consensus metadata
I20260809 04:56:31.036695 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 5078
W20260809 04:56:31.063665 4950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.243.193:38137 (error 108)
I20260809 04:56:31.064443 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 5301
I20260809 04:56:31.066504 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:31.065202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38137 (local address 127.0.0.1:41700)
0809 04:56:31.066061 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:31.066274 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":633,"threads_started":1}
W20260809 04:56:31.066793 4950 catalog_manager.cc:4584] TS 6d2ef1a6881a4a52abe752f4579baf1e (127.29.243.193:38137): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8fe0d61d34ac4f13892aa6b425b02f40: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38137: connect: Connection refused (error 111)
Aug 09 04:56:31 dist-test-slave-21b9 krb5kdc[4907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:56:31.081966 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:31.065275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52412 (local address 127.29.243.194:40431)
0809 04:56:31.065484 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:31.065490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:31.065601 (+ 111us) server_negotiation.cc:407] Connection header received
0809 04:56:31.065782 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:31.065789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:31.065889 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:31.065993 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:31.067799 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:31.068562 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:31.069810 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:31.070217 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:31.070330 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:31.074750 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:31.074770 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:31.074791 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:31.074848 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:31.078738 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:31.079694 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:31.079705 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:31.079709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:31.079889 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:31.080298 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:31.080304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:31.080308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:31.080819 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:31.081156 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:31.081332 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 04:56:31.081581 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:56:31.082724 5506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:31.120911 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:31.120315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38137 (local address 127.0.0.1:41706)
0809 04:56:31.120582 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:31.120713 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 04:56:31.122231 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:31.121788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38137 (local address 127.0.0.1:41712)
0809 04:56:31.121958 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:31.122053 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 04:56:31.174122 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 5301
I20260809 04:56:31.183580 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:31.182834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38137 (local address 127.0.0.1:41724)
0809 04:56:31.183199 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:31.183358 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
I20260809 04:56:31.184685 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:31.184316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38137 (local address 127.0.0.1:41738)
0809 04:56:31.184469 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:31.184544 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 04:56:31.210770 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 5521
Aug 09 04:56:31 dist-test-slave-21b9 krb5kdc[4907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:56:31.229084 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:31.211683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55266 (local address 127.29.243.195:40609)
0809 04:56:31.211861 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:31.211865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:31.211931 (+ 66us) server_negotiation.cc:407] Connection header received
0809 04:56:31.212152 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:31.212156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:31.212228 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:31.212318 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:31.214024 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:31.214804 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:31.216985 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:31.217476 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:31.217579 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:31.222962 (+ 5383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:31.222979 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:31.222998 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:31.223040 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:31.226268 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:31.226941 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:31.226949 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:31.226951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:31.227094 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:31.227508 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:31.227511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:31.227513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:31.227914 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:31.228215 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:31.228529 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 04:56:31.228745 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:56:31.229775 5704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:31.266089 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:31.265511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38137 (local address 127.0.0.1:41742)
0809 04:56:31.265775 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:31.265893 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 04:56:31.267387 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:31.266833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38137 (local address 127.0.0.1:41744)
0809 04:56:31.267083 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:31.267174 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260809 04:56:31.353525 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 5521
I20260809 04:56:31.382138 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 4927
I20260809 04:56:31.383664 5024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:31.494977 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:31.494266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38137 (local address 127.0.0.1:41746)
0809 04:56:31.494650 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:31.494788 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221}
I20260809 04:56:31.495004 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 4927
2026-08-09T04:56:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4545 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T04:56:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:31Z Disabled control of system clock
I20260809 04:56:31.573882 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:32773
--webserver_interface=127.29.243.254
--webserver_port=0
--builtin_ntp_servers=127.29.243.212:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:32773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 04:56:31.903914 5804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:31.904186 5804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:31.904278 5804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:31.911350 5804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:31.911448 5804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:31.911485 5804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:31.911542 5804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:31.911585 5804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:31.920411 5804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:32773
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:32773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:31.921717 5804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:31.923511 5804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:31.935089 5812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:31.935055 5813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:31.935118 5818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:31.935966 5804 server_base.cc:1034] running on GCE node
I20260809 04:56:31.936712 5804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:31.938095 5804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:31.939337 5804 hybrid_clock.cc:648] HybridClock initialized: now 1786251391939265 us; error 49 us; skew 500 ppm
I20260809 04:56:31.939683 5804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:56:31.942049 5804 webserver.cc:460] Webserver started at http://127.29.243.254:35291/ using document root <none> and password file <none>
I20260809 04:56:31.942749 5804 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:31.942848 5804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:31.943204 5804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:31.946206 5804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "5210b6f9ee3b4ec7b9606f63df16b50f"
format_stamp: "Formatted at 2026-08-09 04:56:31 on dist-test-slave-21b9"
server_key: "0ccd7492fd2edb755c1d053535ad74c8"
server_key_iv: "50d539d58f5b9a8b088c2ed4c4c8814b"
server_key_version: "encryptionkey@0"
I20260809 04:56:31.947018 5804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "5210b6f9ee3b4ec7b9606f63df16b50f"
format_stamp: "Formatted at 2026-08-09 04:56:31 on dist-test-slave-21b9"
server_key: "0ccd7492fd2edb755c1d053535ad74c8"
server_key_iv: "50d539d58f5b9a8b088c2ed4c4c8814b"
server_key_version: "encryptionkey@0"
I20260809 04:56:31.952339 5804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:56:31.955906 5827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:31.957286 5804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:56:31.957464 5804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "5210b6f9ee3b4ec7b9606f63df16b50f"
format_stamp: "Formatted at 2026-08-09 04:56:31 on dist-test-slave-21b9"
server_key: "0ccd7492fd2edb755c1d053535ad74c8"
server_key_iv: "50d539d58f5b9a8b088c2ed4c4c8814b"
server_key_version: "encryptionkey@0"
I20260809 04:56:31.957685 5804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:31.964011 5804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:31.964805 5804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:31.965143 5804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:31.984475 5804 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:32773
I20260809 04:56:31.984496 5903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:32773 every 8 connection(s)
I20260809 04:56:31.985934 5804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:31.989717 5905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:31.992345 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 5804
I20260809 04:56:31.992560 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:31.993240 30671 external_mini_cluster.cc:1428] Setting key 26e75eb8d704f15f76372f1f1f875ee2
I20260809 04:56:32.001636 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:31.995342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43000 (local address 127.29.243.254:32773)
0809 04:56:31.996338 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:31.996348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:31.996393 (+ 45us) server_negotiation.cc:407] Connection header received
0809 04:56:31.998080 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:31.998104 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:31.998547 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:31.998906 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:32.000025 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:32.000032 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:32.000045 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:32.000067 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:32.000136 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:32.000157 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:32.000351 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:32.000456 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 04:56:32.000691 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":353,"threads_started":1}
I20260809 04:56:32.001972 5905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f: Bootstrap starting.
I20260809 04:56:32.005079 5905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:32.006086 5905 log.cc:824] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:32.009097 5905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f: No bootstrap required, opened a new log
I20260809 04:56:32.018111 5905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5210b6f9ee3b4ec7b9606f63df16b50f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32773 } }
I20260809 04:56:32.018491 5905 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:32.018594 5905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5210b6f9ee3b4ec7b9606f63df16b50f, State: Initialized, Role: FOLLOWER
I20260809 04:56:32.019133 5905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5210b6f9ee3b4ec7b9606f63df16b50f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32773 } }
I20260809 04:56:32.019408 5905 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:32.019520 5905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:32.019667 5905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:32.022032 5905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5210b6f9ee3b4ec7b9606f63df16b50f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32773 } }
I20260809 04:56:32.022505 5905 leader_election.cc:302] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5210b6f9ee3b4ec7b9606f63df16b50f; no voters:
I20260809 04:56:32.023078 5905 leader_election.cc:288] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:32.023238 5912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:32.024101 5912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [term 1 LEADER]: Becoming Leader. State: Replica: 5210b6f9ee3b4ec7b9606f63df16b50f, State: Running, Role: LEADER
I20260809 04:56:32.024689 5912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5210b6f9ee3b4ec7b9606f63df16b50f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32773 } }
I20260809 04:56:32.025242 5905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:32.027740 5913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5210b6f9ee3b4ec7b9606f63df16b50f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5210b6f9ee3b4ec7b9606f63df16b50f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32773 } } }
I20260809 04:56:32.028151 5913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:32.029122 5914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5210b6f9ee3b4ec7b9606f63df16b50f. Latest consensus state: current_term: 1 leader_uuid: "5210b6f9ee3b4ec7b9606f63df16b50f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5210b6f9ee3b4ec7b9606f63df16b50f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32773 } } }
I20260809 04:56:32.029539 5914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:32.030764 5923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:32.035449 5923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:32.044390 5923 catalog_manager.cc:1347] Generated new cluster ID: 2b139f250ba74b2f885b2ee2c6c3fe4b
I20260809 04:56:32.044523 5923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:32.064210 5923 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:32.064649 5923 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:32.075416 5923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f: Generated new TSK 0
I20260809 04:56:32.076243 5923 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:56:32.089759 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32773
--builtin_ntp_servers=127.29.243.212:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 04:56:32.430631 5944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:32.430924 5944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:32.431051 5944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:32.438174 5944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:32.438287 5944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:32.438448 5944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:56:32.447540 5944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:32.448884 5944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:32.450719 5944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:32.464046 5949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:32.464097 5953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:32.464121 5951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:32.468732 5944 server_base.cc:1034] running on GCE node
I20260809 04:56:32.469321 5944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:32.470351 5944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:32.471560 5944 hybrid_clock.cc:648] HybridClock initialized: now 1786251392471502 us; error 48 us; skew 500 ppm
I20260809 04:56:32.471858 5944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:56:32.474092 5944 webserver.cc:460] Webserver started at http://127.29.243.193:37585/ using document root <none> and password file <none>
I20260809 04:56:32.474773 5944 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:32.474860 5944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:32.475147 5944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:32.478084 5944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "5c4e7d9c86504d4a822c20ea08e27851"
format_stamp: "Formatted at 2026-08-09 04:56:32 on dist-test-slave-21b9"
server_key: "bed9e4f83759bf06d468fdd34a48a924"
server_key_iv: "877b0c3fa3fc9dd29ef017a97b8d818b"
server_key_version: "encryptionkey@0"
I20260809 04:56:32.478825 5944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "5c4e7d9c86504d4a822c20ea08e27851"
format_stamp: "Formatted at 2026-08-09 04:56:32 on dist-test-slave-21b9"
server_key: "bed9e4f83759bf06d468fdd34a48a924"
server_key_iv: "877b0c3fa3fc9dd29ef017a97b8d818b"
server_key_version: "encryptionkey@0"
I20260809 04:56:32.484072 5944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 04:56:32.487478 5962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:32.488792 5944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:56:32.488950 5944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "5c4e7d9c86504d4a822c20ea08e27851"
format_stamp: "Formatted at 2026-08-09 04:56:32 on dist-test-slave-21b9"
server_key: "bed9e4f83759bf06d468fdd34a48a924"
server_key_iv: "877b0c3fa3fc9dd29ef017a97b8d818b"
server_key_version: "encryptionkey@0"
I20260809 04:56:32.489163 5944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:32.494931 5944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:32.495745 5944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:32.495994 5944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:32.496901 5944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:32.498312 5944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:32.498410 5944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:32.498521 5944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:32.498591 5944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:32.530015 5944 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:38905
I20260809 04:56:32.530133 6129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:38905 every 8 connection(s)
I20260809 04:56:32.531484 5944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:56:32.533883 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 5944
I20260809 04:56:32.534092 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:56:32.534816 30671 external_mini_cluster.cc:1428] Setting key 94f3ced21d73952cfe42d7f96062830e
I20260809 04:56:32.538023 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32773
--builtin_ntp_servers=127.29.243.212:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 04:56:32.540700 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:32.534127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:45805 (local address 127.29.243.254:32773)
0809 04:56:32.534613 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:32.534620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:32.536024 (+ 1404us) server_negotiation.cc:407] Connection header received
0809 04:56:32.538225 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:32.538232 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:32.538295 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:32.538404 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:32.539943 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:32.539949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:32.539952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:32.539971 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:32.540022 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:32.540037 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:32.540171 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:32.540425 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 04:56:32.540481 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":262,"threads_started":1}
I20260809 04:56:32.541847 6135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:32.534688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:32773 (local address 127.29.243.193:45805)
0809 04:56:32.535842 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:32.535889 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:32.537860 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:32.538668 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:32.538679 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:32.538800 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:32.539565 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:56:32.539568 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:32.539619 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:56:32.539733 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:32.540147 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:32.540158 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:32.540165 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 04:56:32.540442 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 04:56:32.540696 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":316,"threads_started":1}
I20260809 04:56:32.543354 6130 heartbeater.cc:347] Connected to a master server at 127.29.243.254:32773
I20260809 04:56:32.543682 6130 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:32.544396 6130 heartbeater.cc:511] Master 127.29.243.254:32773 requested a full tablet report, sending...
I20260809 04:56:32.546641 5852 ts_manager.cc:194] Registered new tserver with Master: 5c4e7d9c86504d4a822c20ea08e27851 (127.29.243.193:38905)
W20260809 04:56:32.874115 6136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:32.874409 6136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:32.874519 6136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:32.881690 6136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:32.881786 6136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:32.881896 6136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:56:32.890800 6136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:32.892091 6136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:32.893970 6136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:32.906541 6145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:32.906571 6144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:32.907039 6147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:32.907294 6136 server_base.cc:1034] running on GCE node
I20260809 04:56:32.907856 6136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:32.908739 6136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:32.909956 6136 hybrid_clock.cc:648] HybridClock initialized: now 1786251392909910 us; error 36 us; skew 500 ppm
I20260809 04:56:32.910261 6136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:56:32.912544 6136 webserver.cc:460] Webserver started at http://127.29.243.194:39219/ using document root <none> and password file <none>
I20260809 04:56:32.913236 6136 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:32.913326 6136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:32.913654 6136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:32.916496 6136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "3169c5014d524c958ac1ff58427af7c8"
format_stamp: "Formatted at 2026-08-09 04:56:32 on dist-test-slave-21b9"
server_key: "78af2133989cb3dc6824a3e3f8d52e85"
server_key_iv: "16a7f9eb0df1b0f154221c6966f7a22a"
server_key_version: "encryptionkey@0"
I20260809 04:56:32.917246 6136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "3169c5014d524c958ac1ff58427af7c8"
format_stamp: "Formatted at 2026-08-09 04:56:32 on dist-test-slave-21b9"
server_key: "78af2133989cb3dc6824a3e3f8d52e85"
server_key_iv: "16a7f9eb0df1b0f154221c6966f7a22a"
server_key_version: "encryptionkey@0"
I20260809 04:56:32.922348 6136 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 04:56:32.925916 6160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:32.927332 6136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:32.927484 6136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "3169c5014d524c958ac1ff58427af7c8"
format_stamp: "Formatted at 2026-08-09 04:56:32 on dist-test-slave-21b9"
server_key: "78af2133989cb3dc6824a3e3f8d52e85"
server_key_iv: "16a7f9eb0df1b0f154221c6966f7a22a"
server_key_version: "encryptionkey@0"
I20260809 04:56:32.927686 6136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:32.933375 6136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:32.934132 6136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:32.934374 6136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:32.935290 6136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:32.936764 6136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:32.936851 6136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:32.936968 6136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:32.937042 6136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:32.965844 6136 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:46001
I20260809 04:56:32.965898 6336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:46001 every 8 connection(s)
I20260809 04:56:32.967118 6136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:56:32.972177 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 6136
I20260809 04:56:32.972353 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:56:32.972777 30671 external_mini_cluster.cc:1428] Setting key 52850b19b2b699f6420e89c9d2ff04af
I20260809 04:56:32.975526 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:32.969894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:46113 (local address 127.29.243.254:32773)
0809 04:56:32.970147 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:32.970152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:32.971566 (+ 1414us) server_negotiation.cc:407] Connection header received
0809 04:56:32.973289 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:32.973293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:32.973350 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:32.973441 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:32.974790 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:32.974797 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:32.974801 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:32.974821 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:32.974862 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:32.974893 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:32.975016 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:32.975152 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 04:56:32.975213 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 04:56:32.975780 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32773
--builtin_ntp_servers=127.29.243.212:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 04:56:32.976450 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:32.970421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:32773 (local address 127.29.243.194:46113)
0809 04:56:32.971394 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:32.971432 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:32.973022 (+ 1590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:32.973669 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:32.973677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:32.973764 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:32.974432 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:56:32.974436 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:32.974498 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:56:32.974613 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:32.974983 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:32.974989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:32.974994 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:56:32.975197 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 04:56:32.975436 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":301,"threads_started":1}
I20260809 04:56:32.980329 6340 heartbeater.cc:347] Connected to a master server at 127.29.243.254:32773
I20260809 04:56:32.980629 6340 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:32.981241 6340 heartbeater.cc:511] Master 127.29.243.254:32773 requested a full tablet report, sending...
I20260809 04:56:32.982667 5852 ts_manager.cc:194] Registered new tserver with Master: 3169c5014d524c958ac1ff58427af7c8 (127.29.243.194:46001)
W20260809 04:56:33.315982 6349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:33.316268 6349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:33.316392 6349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:33.323575 6349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:33.323675 6349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:33.323822 6349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:56:33.332998 6349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:33.334264 6349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:33.336066 6349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:33.348981 6357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:33.348935 6356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:33.349014 6360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:33.349838 6349 server_base.cc:1034] running on GCE node
I20260809 04:56:33.350445 6349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:33.351446 6349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:33.352672 6349 hybrid_clock.cc:648] HybridClock initialized: now 1786251393352606 us; error 53 us; skew 500 ppm
I20260809 04:56:33.353008 6349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:56:33.355342 6349 webserver.cc:460] Webserver started at http://127.29.243.195:37351/ using document root <none> and password file <none>
I20260809 04:56:33.356070 6349 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:33.356164 6349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:33.356474 6349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:33.359372 6349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "d27ad1ebf0d74c19ae57be7e415101ae"
format_stamp: "Formatted at 2026-08-09 04:56:33 on dist-test-slave-21b9"
server_key: "696c6814beaf158ab009cba7583a82c4"
server_key_iv: "538d1bcb648fe6a60e8097ba0d2ba22e"
server_key_version: "encryptionkey@0"
I20260809 04:56:33.360175 6349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "d27ad1ebf0d74c19ae57be7e415101ae"
format_stamp: "Formatted at 2026-08-09 04:56:33 on dist-test-slave-21b9"
server_key: "696c6814beaf158ab009cba7583a82c4"
server_key_iv: "538d1bcb648fe6a60e8097ba0d2ba22e"
server_key_version: "encryptionkey@0"
I20260809 04:56:33.365360 6349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 04:56:33.368884 6367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:33.370231 6349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:56:33.370389 6349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "d27ad1ebf0d74c19ae57be7e415101ae"
format_stamp: "Formatted at 2026-08-09 04:56:33 on dist-test-slave-21b9"
server_key: "696c6814beaf158ab009cba7583a82c4"
server_key_iv: "538d1bcb648fe6a60e8097ba0d2ba22e"
server_key_version: "encryptionkey@0"
I20260809 04:56:33.370590 6349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:33.376197 6349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:33.376955 6349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:33.377202 6349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:33.378082 6349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:33.379458 6349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:33.379552 6349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:33.379660 6349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:33.379730 6349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:33.410575 6349 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:43187
I20260809 04:56:33.410624 6543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:43187 every 8 connection(s)
I20260809 04:56:33.411919 6349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:56:33.412536 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 6349
I20260809 04:56:33.412716 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:56:33.413270 30671 external_mini_cluster.cc:1428] Setting key 4346423e94853fa09a23e18d7210a8ee
I20260809 04:56:33.420416 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.414730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:46465 (local address 127.29.243.254:32773)
0809 04:56:33.414920 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:33.414927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:33.416644 (+ 1717us) server_negotiation.cc:407] Connection header received
0809 04:56:33.418302 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:33.418306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:33.418368 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:33.418459 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:33.419741 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:33.419745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:33.419755 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:33.419775 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:33.419810 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:33.419822 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:33.419926 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:33.420126 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 04:56:33.420185 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:56:33.421265 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.415339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:32773 (local address 127.29.243.195:46465)
0809 04:56:33.416455 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:33.416519 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:33.418025 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:33.418674 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:33.418682 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:33.418773 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:33.419443 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:56:33.419445 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:33.419485 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:56:33.419581 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:33.419907 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:33.419912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:33.419917 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:56:33.420136 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 04:56:33.420339 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":333,"threads_started":1}
I20260809 04:56:33.422555 6544 heartbeater.cc:347] Connected to a master server at 127.29.243.254:32773
I20260809 04:56:33.422823 6544 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:33.423439 6544 heartbeater.cc:511] Master 127.29.243.254:32773 requested a full tablet report, sending...
I20260809 04:56:33.424767 5852 ts_manager.cc:194] Registered new tserver with Master: d27ad1ebf0d74c19ae57be7e415101ae (127.29.243.195:43187)
I20260809 04:56:33.428601 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:56:33.435461 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.433160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43010 (local address 127.29.243.254:32773)
0809 04:56:33.433341 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:33.433349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:33.433762 (+ 413us) server_negotiation.cc:407] Connection header received
0809 04:56:33.433896 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:33.433900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:33.433957 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:33.434026 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:33.434836 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:33.434841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:33.434844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:33.434858 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:33.434888 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:33.434899 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:33.435003 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:33.435061 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 04:56:33.435150 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:56:33.439208 5852 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:33.445998 5852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:33.462921 6561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.458023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38905 (local address 127.0.0.1:42454)
0809 04:56:33.458731 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:33.458772 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:33.458916 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:33.460591 (+ 1675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:33.460600 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:33.460695 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:33.461537 (+ 842us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:56:33.461541 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:33.461572 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:56:33.461658 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:33.462165 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:33.462176 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:33.462185 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:56:33.462372 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 04:56:33.462464 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":285,"threads_started":1}
I20260809 04:56:33.463949 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.458225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42454 (local address 127.29.243.193:38905)
0809 04:56:33.458748 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:33.458757 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:33.458871 (+ 114us) server_negotiation.cc:407] Connection header received
0809 04:56:33.459088 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:33.459098 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:33.459395 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:33.459556 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:33.461855 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:33.461864 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:33.461881 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:33.461911 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:33.461973 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:33.461993 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:33.462114 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:33.463497 (+ 1383us) server_negotiation.cc:299] Negotiation successful
0809 04:56:33.463609 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":241,"threads_started":1}
I20260809 04:56:33.471692 6561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.459080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:43187 (local address 127.0.0.1:46996)
0809 04:56:33.464328 (+ 5248us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:33.464347 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:33.464441 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:33.469060 (+ 4619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:33.469085 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:33.469149 (+ 64us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:33.469695 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:56:33.469698 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:33.469714 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:56:33.469760 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:33.471135 (+ 1375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:33.471140 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:33.471145 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:56:33.471227 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 04:56:33.471302 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5152,"thread_start_us":9531,"threads_started":1}
I20260809 04:56:33.471714 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.459319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:46001 (local address 127.0.0.1:57714)
0809 04:56:33.468171 (+ 8852us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:33.468195 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:33.468299 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:33.469673 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:33.469678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:33.469701 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:33.470320 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:56:33.470324 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:33.470353 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:56:33.470400 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:33.470938 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:33.470943 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:33.470948 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:56:33.471093 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 04:56:33.471147 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8722,"thread_start_us":8574,"threads_started":1}
I20260809 04:56:33.471774 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.459897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57714 (local address 127.29.243.194:46001)
0809 04:56:33.467725 (+ 7828us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:33.467734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:33.468436 (+ 702us) server_negotiation.cc:407] Connection header received
0809 04:56:33.468518 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:33.468527 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:33.468861 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:33.469013 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:33.470580 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:33.470590 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:33.470605 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:33.470633 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:33.470693 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:33.470715 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:33.470875 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:33.471113 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 04:56:33.471218 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7660,"thread_start_us":7529,"threads_started":1}
I20260809 04:56:33.472781 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.459510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46996 (local address 127.29.243.195:43187)
0809 04:56:33.468574 (+ 9064us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:33.468582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:33.468613 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:56:33.468675 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:33.468682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:33.468868 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:33.469462 (+ 594us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:33.470902 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:33.470910 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:33.470924 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:33.470941 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:33.470987 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:33.471002 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:33.472250 (+ 1248us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:33.472323 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 04:56:33.472413 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8888,"thread_start_us":8728,"threads_started":1}
I20260809 04:56:33.473261 6032 tablet_service.cc:1467] Processing CreateTablet for tablet 37de22516c1f4f8187ec10984188e573 (DEFAULT_TABLE table=test-table [id=0e319b8bd2dd495491378dd26253fddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:33.474905 6032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37de22516c1f4f8187ec10984188e573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:33.475569 6225 tablet_service.cc:1467] Processing CreateTablet for tablet 37de22516c1f4f8187ec10984188e573 (DEFAULT_TABLE table=test-table [id=0e319b8bd2dd495491378dd26253fddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:33.476004 6446 tablet_service.cc:1467] Processing CreateTablet for tablet 37de22516c1f4f8187ec10984188e573 (DEFAULT_TABLE table=test-table [id=0e319b8bd2dd495491378dd26253fddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:33.476999 6225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37de22516c1f4f8187ec10984188e573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:33.477277 6446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37de22516c1f4f8187ec10984188e573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:33.488678 6568 tablet_bootstrap.cc:492] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae: Bootstrap starting.
I20260809 04:56:33.490718 6569 tablet_bootstrap.cc:492] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851: Bootstrap starting.
I20260809 04:56:33.493180 6568 tablet_bootstrap.cc:654] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:33.493228 6571 tablet_bootstrap.cc:492] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8: Bootstrap starting.
I20260809 04:56:33.494446 6569 tablet_bootstrap.cc:654] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:33.494536 6568 log.cc:824] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:33.495625 6569 log.cc:824] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:33.496073 6571 tablet_bootstrap.cc:654] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:33.497205 6571 log.cc:824] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:33.497766 6568 tablet_bootstrap.cc:492] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae: No bootstrap required, opened a new log
I20260809 04:56:33.498106 6568 ts_tablet_manager.cc:1397] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 04:56:33.498649 6569 tablet_bootstrap.cc:492] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851: No bootstrap required, opened a new log
I20260809 04:56:33.498927 6569 ts_tablet_manager.cc:1397] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 04:56:33.499928 6571 tablet_bootstrap.cc:492] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8: No bootstrap required, opened a new log
I20260809 04:56:33.500288 6571 ts_tablet_manager.cc:1397] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 04:56:33.507230 6571 raft_consensus.cc:348] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4e7d9c86504d4a822c20ea08e27851" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38905 } } peers { permanent_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43187 } } peers { permanent_uuid: "3169c5014d524c958ac1ff58427af7c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46001 } }
I20260809 04:56:33.507643 6571 raft_consensus.cc:374] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:33.507807 6571 raft_consensus.cc:729] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3169c5014d524c958ac1ff58427af7c8, State: Initialized, Role: FOLLOWER
I20260809 04:56:33.508420 6568 raft_consensus.cc:348] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4e7d9c86504d4a822c20ea08e27851" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38905 } } peers { permanent_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43187 } } peers { permanent_uuid: "3169c5014d524c958ac1ff58427af7c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46001 } }
I20260809 04:56:33.508693 6571 consensus_queue.cc:260] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4e7d9c86504d4a822c20ea08e27851" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38905 } } peers { permanent_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43187 } } peers { permanent_uuid: "3169c5014d524c958ac1ff58427af7c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46001 } }
I20260809 04:56:33.508930 6568 raft_consensus.cc:374] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:33.509091 6568 raft_consensus.cc:729] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d27ad1ebf0d74c19ae57be7e415101ae, State: Initialized, Role: FOLLOWER
I20260809 04:56:33.509136 6569 raft_consensus.cc:348] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4e7d9c86504d4a822c20ea08e27851" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38905 } } peers { permanent_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43187 } } peers { permanent_uuid: "3169c5014d524c958ac1ff58427af7c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46001 } }
I20260809 04:56:33.509617 6569 raft_consensus.cc:374] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:33.509758 6569 raft_consensus.cc:729] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4e7d9c86504d4a822c20ea08e27851, State: Initialized, Role: FOLLOWER
I20260809 04:56:33.509932 6340 heartbeater.cc:503] Master 127.29.243.254:32773 was elected leader, sending a full tablet report...
I20260809 04:56:33.510481 6571 ts_tablet_manager.cc:1428] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260809 04:56:33.510505 6569 consensus_queue.cc:260] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4e7d9c86504d4a822c20ea08e27851" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38905 } } peers { permanent_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43187 } } peers { permanent_uuid: "3169c5014d524c958ac1ff58427af7c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46001 } }
I20260809 04:56:33.510727 6568 consensus_queue.cc:260] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4e7d9c86504d4a822c20ea08e27851" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38905 } } peers { permanent_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43187 } } peers { permanent_uuid: "3169c5014d524c958ac1ff58427af7c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46001 } }
I20260809 04:56:33.511870 6130 heartbeater.cc:503] Master 127.29.243.254:32773 was elected leader, sending a full tablet report...
I20260809 04:56:33.512197 6544 heartbeater.cc:503] Master 127.29.243.254:32773 was elected leader, sending a full tablet report...
I20260809 04:56:33.512372 6569 ts_tablet_manager.cc:1428] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260809 04:56:33.512809 6568 ts_tablet_manager.cc:1428] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae: Time spent starting tablet: real 0.015s user 0.008s sys 0.005s
W20260809 04:56:33.533674 6131 tablet.cc:2369] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:56:33.589252 6582 raft_consensus.cc:482] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:33.589591 6582 raft_consensus.cc:504] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4e7d9c86504d4a822c20ea08e27851" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38905 } } peers { permanent_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43187 } } peers { permanent_uuid: "3169c5014d524c958ac1ff58427af7c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46001 } }
I20260809 04:56:33.591631 6582 leader_election.cc:288] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c4e7d9c86504d4a822c20ea08e27851 (127.29.243.193:38905), 3169c5014d524c958ac1ff58427af7c8 (127.29.243.194:46001)
I20260809 04:56:33.594230 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.591783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38905 (local address 127.29.243.195:55413)
0809 04:56:33.592020 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:33.592042 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:33.592146 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:33.592691 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:33.592696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:33.592726 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:33.593355 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:56:33.593393 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:33.593412 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:56:33.593468 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:33.593789 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:33.593795 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:33.593800 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:56:33.593918 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 04:56:33.593979 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 04:56:33.594244 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.592126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:55413 (local address 127.29.243.193:38905)
0809 04:56:33.592370 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:33.592376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:33.592396 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:56:33.592456 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:33.592460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:33.592536 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:33.592646 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:33.593623 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:33.593627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:33.593631 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:33.593646 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:33.593682 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:33.593694 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:33.593793 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:33.593924 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 04:56:33.593983 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260809 04:56:33.594918 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.592162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:53723 (local address 127.29.243.194:46001)
0809 04:56:33.592325 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:33.592331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:33.593061 (+ 730us) server_negotiation.cc:407] Connection header received
0809 04:56:33.593136 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:33.593141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:33.593204 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:33.593305 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:33.594180 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:33.594184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:33.594187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:33.594203 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:33.594233 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:33.594245 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:33.594356 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:33.594627 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 04:56:33.594697 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:56:33.595067 6584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.592137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:46001 (local address 127.29.243.195:53723)
0809 04:56:33.592955 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:33.592970 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:33.593055 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:33.593355 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:33.593360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:33.593383 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:33.593990 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:56:33.593993 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:33.594005 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:56:33.594040 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:33.594443 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:33.594447 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:33.594451 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:56:33.594603 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 04:56:33.594651 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":479,"threads_started":1}
I20260809 04:56:33.595367 6063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37de22516c1f4f8187ec10984188e573" candidate_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4e7d9c86504d4a822c20ea08e27851" is_pre_election: true
I20260809 04:56:33.595950 6063 raft_consensus.cc:2454] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d27ad1ebf0d74c19ae57be7e415101ae in term 0.
I20260809 04:56:33.595989 6266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37de22516c1f4f8187ec10984188e573" candidate_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3169c5014d524c958ac1ff58427af7c8" is_pre_election: true
I20260809 04:56:33.596529 6266 raft_consensus.cc:2454] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d27ad1ebf0d74c19ae57be7e415101ae in term 0.
I20260809 04:56:33.596555 6372 leader_election.cc:302] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c4e7d9c86504d4a822c20ea08e27851, d27ad1ebf0d74c19ae57be7e415101ae; no voters:
I20260809 04:56:33.596880 6582 raft_consensus.cc:2781] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:33.597016 6582 raft_consensus.cc:482] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:33.597124 6582 raft_consensus.cc:3037] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:33.599416 6582 raft_consensus.cc:504] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4e7d9c86504d4a822c20ea08e27851" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38905 } } peers { permanent_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43187 } } peers { permanent_uuid: "3169c5014d524c958ac1ff58427af7c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46001 } }
I20260809 04:56:33.599989 6582 leader_election.cc:288] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [CANDIDATE]: Term 1 election: Requested vote from peers 5c4e7d9c86504d4a822c20ea08e27851 (127.29.243.193:38905), 3169c5014d524c958ac1ff58427af7c8 (127.29.243.194:46001)
I20260809 04:56:33.600430 6063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37de22516c1f4f8187ec10984188e573" candidate_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4e7d9c86504d4a822c20ea08e27851"
I20260809 04:56:33.600493 6266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37de22516c1f4f8187ec10984188e573" candidate_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3169c5014d524c958ac1ff58427af7c8"
I20260809 04:56:33.600615 6063 raft_consensus.cc:3037] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:33.600696 6266 raft_consensus.cc:3037] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:33.602793 6063 raft_consensus.cc:2454] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d27ad1ebf0d74c19ae57be7e415101ae in term 1.
I20260809 04:56:33.602881 6266 raft_consensus.cc:2454] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d27ad1ebf0d74c19ae57be7e415101ae in term 1.
I20260809 04:56:33.603230 6372 leader_election.cc:302] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c4e7d9c86504d4a822c20ea08e27851, d27ad1ebf0d74c19ae57be7e415101ae; no voters:
I20260809 04:56:33.603549 6582 raft_consensus.cc:2781] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:33.603965 6582 raft_consensus.cc:686] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 1 LEADER]: Becoming Leader. State: Replica: d27ad1ebf0d74c19ae57be7e415101ae, State: Running, Role: LEADER
I20260809 04:56:33.604391 6582 consensus_queue.cc:237] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4e7d9c86504d4a822c20ea08e27851" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38905 } } peers { permanent_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43187 } } peers { permanent_uuid: "3169c5014d524c958ac1ff58427af7c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46001 } }
I20260809 04:56:33.609201 5852 catalog_manager.cc:5495] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae reported cstate change: term changed from 0 to 1, leader changed from <none> to d27ad1ebf0d74c19ae57be7e415101ae (127.29.243.195). New cstate: current_term: 1 leader_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c4e7d9c86504d4a822c20ea08e27851" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d27ad1ebf0d74c19ae57be7e415101ae" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3169c5014d524c958ac1ff58427af7c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46001 } health_report { overall_health: UNKNOWN } } }
I20260809 04:56:33.630465 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.628746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.29.243.195:43187)
0809 04:56:33.628905 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:33.628909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:33.628926 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:56:33.629043 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:33.629047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:33.629136 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:33.629215 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:33.629984 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:33.629989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:33.629992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:33.630007 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:33.630037 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:33.630048 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:33.630145 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:33.630198 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 04:56:33.630242 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:56:33.638536 6063 raft_consensus.cc:1261] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851 [term 1 FOLLOWER]: Refusing update from remote peer d27ad1ebf0d74c19ae57be7e415101ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:33.638563 6266 raft_consensus.cc:1261] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8 [term 1 FOLLOWER]: Refusing update from remote peer d27ad1ebf0d74c19ae57be7e415101ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:33.639431 6585 consensus_queue.cc:1035] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c4e7d9c86504d4a822c20ea08e27851" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:33.639750 6582 consensus_queue.cc:1035] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "3169c5014d524c958ac1ff58427af7c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:33.649335 6592 mvcc.cc:205] Tried to move back new op lower bound from 7316485708333953024 to 7316485708207230976. Current Snapshot: MvccSnapshot[applied={T|T < 7316485708333953024}]
I20260809 04:56:33.666541 5852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:33.666853 5852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:33.670047 6593 mvcc.cc:205] Tried to move back new op lower bound from 7316485708333953024 to 7316485708207230976. Current Snapshot: MvccSnapshot[applied={T|T < 7316485708333953024}]
I20260809 04:56:33.671075 5852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5210b6f9ee3b4ec7b9606f63df16b50f: Sending DeleteTablet for 3 replicas of tablet 37de22516c1f4f8187ec10984188e573
I20260809 04:56:33.672256 6032 tablet_service.cc:1514] Processing DeleteTablet for tablet 37de22516c1f4f8187ec10984188e573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:33 UTC) from {username='slave'} at 127.0.0.1:42454
I20260809 04:56:33.674032 6446 tablet_service.cc:1514] Processing DeleteTablet for tablet 37de22516c1f4f8187ec10984188e573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:33 UTC) from {username='slave'} at 127.0.0.1:46996
I20260809 04:56:33.674639 6225 tablet_service.cc:1514] Processing DeleteTablet for tablet 37de22516c1f4f8187ec10984188e573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:33 UTC) from {username='slave'} at 127.0.0.1:57714
I20260809 04:56:33.675494 6605 tablet_replica.cc:330] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8: stopping tablet replica
I20260809 04:56:33.675989 6605 raft_consensus.cc:2229] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:33.676457 6605 raft_consensus.cc:2258] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:33.677386 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 5944
I20260809 04:56:33.687884 6605 ts_tablet_manager.cc:1905] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:33.694695 6603 tablet_replica.cc:330] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851: stopping tablet replica
I20260809 04:56:33.695353 6603 raft_consensus.cc:2229] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:33.696286 6603 raft_consensus.cc:2258] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:33.696714 6604 tablet_replica.cc:330] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae: stopping tablet replica
I20260809 04:56:33.697184 6604 raft_consensus.cc:2229] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:33.698491 6604 raft_consensus.cc:2258] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:33.700321 6605 ts_tablet_manager.cc:1918] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:33.700541 6603 ts_tablet_manager.cc:1905] T 37de22516c1f4f8187ec10984188e573 P 5c4e7d9c86504d4a822c20ea08e27851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:33.701334 6605 log.cc:1196] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/37de22516c1f4f8187ec10984188e573
I20260809 04:56:33.701936 6605 ts_tablet_manager.cc:1939] T 37de22516c1f4f8187ec10984188e573 P 3169c5014d524c958ac1ff58427af7c8: Deleting consensus metadata
I20260809 04:56:33.703794 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.679611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42456 (local address 127.29.243.193:38905)
0809 04:56:33.679805 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:33.679812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:33.700983 (+ 21171us) server_negotiation.cc:407] Connection header received
0809 04:56:33.701283 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:33.701289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:33.701381 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:33.701493 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:33.702956 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:33.702963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:33.702967 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:33.702988 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:33.703028 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:33.703044 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:33.703145 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:33.703448 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 04:56:33.703516 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:56:33.703828 5829 catalog_manager.cc:4841] TS 3169c5014d524c958ac1ff58427af7c8 (127.29.243.194:46001): tablet 37de22516c1f4f8187ec10984188e573 (table test-table [id=0e319b8bd2dd495491378dd26253fddf]) successfully deleted
I20260809 04:56:33.704212 6604 ts_tablet_manager.cc:1905] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:33.704790 6122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:33.711462 6604 ts_tablet_manager.cc:1918] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:33.711695 6604 log.cc:1196] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/37de22516c1f4f8187ec10984188e573
I20260809 04:56:33.712338 6604 ts_tablet_manager.cc:1939] T 37de22516c1f4f8187ec10984188e573 P d27ad1ebf0d74c19ae57be7e415101ae: Deleting consensus metadata
I20260809 04:56:33.714120 5830 catalog_manager.cc:4841] TS d27ad1ebf0d74c19ae57be7e415101ae (127.29.243.195:43187): tablet 37de22516c1f4f8187ec10984188e573 (table test-table [id=0e319b8bd2dd495491378dd26253fddf]) successfully deleted
I20260809 04:56:33.793882 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 5944
I20260809 04:56:33.821007 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 6136
W20260809 04:56:33.821100 5830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.243.193:38905 (error 108)
I20260809 04:56:33.823127 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.822620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38905 (local address 127.0.0.1:42462)
0809 04:56:33.822800 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:33.822952 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 04:56:33.823419 5830 catalog_manager.cc:4584] TS 5c4e7d9c86504d4a822c20ea08e27851 (127.29.243.193:38905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37de22516c1f4f8187ec10984188e573: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38905: connect: Connection refused (error 111)
I20260809 04:56:33.824630 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.822242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57718 (local address 127.29.243.194:46001)
0809 04:56:33.822423 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:33.822430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:33.822450 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:56:33.822520 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:33.822525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:33.822638 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:33.822740 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:33.824015 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:33.824021 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:33.824026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:33.824048 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:33.824090 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:33.824105 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:33.824239 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:33.824286 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 04:56:33.824346 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:56:33.825389 6331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:33.877275 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.876702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38905 (local address 127.0.0.1:42466)
0809 04:56:33.876949 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:33.877061 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 04:56:33.878453 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.878032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38905 (local address 127.0.0.1:42472)
0809 04:56:33.878187 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:33.878296 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 04:56:33.912297 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 6136
I20260809 04:56:33.939225 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.938595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38905 (local address 127.0.0.1:42488)
0809 04:56:33.938847 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:33.939005 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 04:56:33.940482 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.940064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38905 (local address 127.0.0.1:42500)
0809 04:56:33.940233 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:33.940323 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 04:56:33.941836 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 6349
I20260809 04:56:33.945029 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:33.942598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47008 (local address 127.29.243.195:43187)
0809 04:56:33.942774 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:33.942780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:33.942818 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:56:33.943056 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:33.943063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:33.943166 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:33.943260 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:33.944319 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:33.944326 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:33.944330 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:33.944352 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:33.944396 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:33.944415 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:33.944574 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:33.944645 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 04:56:33.944703 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:56:33.945811 6538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:34.021207 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:34.020714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38905 (local address 127.0.0.1:42510)
0809 04:56:34.020923 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:34.021028 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 04:56:34.022294 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:34.021865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38905 (local address 127.0.0.1:42512)
0809 04:56:34.022092 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:34.022168 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:38905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 04:56:34.067771 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 6349
I20260809 04:56:34.096753 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 5804
I20260809 04:56:34.098026 5895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:34.204243 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 5804
2026-08-09T04:56:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2710 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 04:56:34.254796 30671 mini_oidc.cc:152] Starting JWKS server
I20260809 04:56:34.258072 30671 webserver.cc:460] Webserver started at https://127.0.0.1:35179/ using document root <none> and password file <none>
I20260809 04:56:34.258426 30671 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:56:34.259075 30671 webserver.cc:460] Webserver started at http://127.0.0.1:33313/ using document root <none> and password file <none>
2026-08-09T04:56:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:34Z Disabled control of system clock
I20260809 04:56:34.291759 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44147
--webserver_interface=127.29.243.254
--webserver_port=0
--builtin_ntp_servers=127.29.243.212:44287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:44147
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35179/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/testchainca.cert with env {}
W20260809 04:56:34.622956 6623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:34.623379 6623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:34.623546 6623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:34.631112 6623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:34.631207 6623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:34.631285 6623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:34.631378 6623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:34.631430 6623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:56:34.631515 6623 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:56:34.631560 6623 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35179/jwks.json
I20260809 04:56:34.641633 6623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:44287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:44147
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35179/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:34.642953 6623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:34.644313 6623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:34.655990 6632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:34.656039 6633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:34.656016 6635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:34.657194 6623 server_base.cc:1034] running on GCE node
I20260809 04:56:34.657968 6623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:34.659478 6623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:34.660696 6623 hybrid_clock.cc:648] HybridClock initialized: now 1786251394660635 us; error 43 us; skew 500 ppm
I20260809 04:56:34.661053 6623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:56:34.663357 6623 webserver.cc:460] Webserver started at http://127.29.243.254:42085/ using document root <none> and password file <none>
I20260809 04:56:34.664059 6623 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:34.664155 6623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:34.664470 6623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:34.667425 6623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "cb868f5053974091994121b5780d35dd"
format_stamp: "Formatted at 2026-08-09 04:56:34 on dist-test-slave-21b9"
server_key: "763ea99a85d2257ae6e7ce4141e37bee"
server_key_iv: "56e1a7b9d78b20e3702904b860da24ce"
server_key_version: "encryptionkey@0"
I20260809 04:56:34.668207 6623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "cb868f5053974091994121b5780d35dd"
format_stamp: "Formatted at 2026-08-09 04:56:34 on dist-test-slave-21b9"
server_key: "763ea99a85d2257ae6e7ce4141e37bee"
server_key_iv: "56e1a7b9d78b20e3702904b860da24ce"
server_key_version: "encryptionkey@0"
I20260809 04:56:34.673348 6623 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:56:34.676905 6645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:34.678220 6623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:56:34.678406 6623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "cb868f5053974091994121b5780d35dd"
format_stamp: "Formatted at 2026-08-09 04:56:34 on dist-test-slave-21b9"
server_key: "763ea99a85d2257ae6e7ce4141e37bee"
server_key_iv: "56e1a7b9d78b20e3702904b860da24ce"
server_key_version: "encryptionkey@0"
I20260809 04:56:34.678619 6623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:34.692713 6623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:34.693500 6623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:34.693799 6623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:34.711656 6623 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:44147
I20260809 04:56:34.711664 6726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:44147 every 8 connection(s)
I20260809 04:56:34.713029 6623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:34.716981 6727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:34.721724 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 6623
I20260809 04:56:34.721937 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:34.722755 30671 external_mini_cluster.cc:1428] Setting key 5c1483b0aff80f50cccde46b6bc951c4
I20260809 04:56:34.728662 6727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd: Bootstrap starting.
I20260809 04:56:34.732438 6727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:34.733652 6727 log.cc:824] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:34.736901 6727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd: No bootstrap required, opened a new log
I20260809 04:56:34.736976 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:34.724755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53040 (local address 127.29.243.254:44147)
0809 04:56:34.725762 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:34.725775 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:34.725821 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:56:34.727643 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:34.727666 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:34.728123 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:34.728438 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:34.729089 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:34.730420 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:34.732401 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:34.733147 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:34.733294 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:34.734556 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:34.734567 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:34.734586 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:34.735259 (+ 673us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:34.735394 (+ 135us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:34.735418 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:34.735511 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:34.735723 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 04:56:34.736041 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":380,"threads_started":1}
I20260809 04:56:34.740671 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:56:34.745513 6727 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb868f5053974091994121b5780d35dd" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44147 } }
I20260809 04:56:34.745833 6727 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:34.745910 6727 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb868f5053974091994121b5780d35dd, State: Initialized, Role: FOLLOWER
I20260809 04:56:34.746438 6727 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb868f5053974091994121b5780d35dd" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44147 } }
I20260809 04:56:34.746642 6727 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:34.746742 6727 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:34.746860 6727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:34.748984 6727 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb868f5053974091994121b5780d35dd" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44147 } }
I20260809 04:56:34.749457 6727 leader_election.cc:302] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb868f5053974091994121b5780d35dd; no voters:
I20260809 04:56:34.750022 6727 leader_election.cc:288] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:34.750209 6732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:34.751206 6732 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [term 1 LEADER]: Becoming Leader. State: Replica: cb868f5053974091994121b5780d35dd, State: Running, Role: LEADER
I20260809 04:56:34.751747 6732 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb868f5053974091994121b5780d35dd" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44147 } }
I20260809 04:56:34.752123 6727 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:34.754709 6734 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb868f5053974091994121b5780d35dd. Latest consensus state: current_term: 1 leader_uuid: "cb868f5053974091994121b5780d35dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb868f5053974091994121b5780d35dd" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44147 } } }
I20260809 04:56:34.754824 6733 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb868f5053974091994121b5780d35dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb868f5053974091994121b5780d35dd" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44147 } } }
I20260809 04:56:34.755167 6733 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:34.755167 6734 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:34.761372 6744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:34.766878 6744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:34.775782 6744 catalog_manager.cc:1347] Generated new cluster ID: a67889ee3470453782de14864b944912
I20260809 04:56:34.775895 6744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:34.801786 6744 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:34.803089 6744 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:34.809104 6744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb868f5053974091994121b5780d35dd: Generated new TSK 0
I20260809 04:56:34.809957 6744 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 04:56:34.854619 6768 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 04:56:34.854852 6768 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 04:56:34.855899 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:34.833847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53044 (local address 127.29.243.254:44147)
0809 04:56:34.834078 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:34.834085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:34.834556 (+ 471us) server_negotiation.cc:407] Connection header received
0809 04:56:34.834962 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:34.834966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:34.835036 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:34.835158 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:56:34.835868 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:34.836634 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:34.838726 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:34.839191 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:34.839316 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:34.839967 (+ 651us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:56:34.855332 (+ 15365us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35179/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 04:56:34.855639 (+ 307us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53044: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35179/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 04:56:34.856033 6730 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53044: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35179/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 04:56:34.858144 6766 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:56:34.833709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44147 (local address 127.0.0.1:53044)
0809 04:56:34.834445 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:34.834465 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:34.834658 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:34.835208 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:34.835214 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:34.835227 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:56:34.835683 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:34.835696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:34.836775 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:34.836781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:34.838570 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:34.838587 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:34.839411 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:34.839419 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:34.839600 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:34.839684 (+ 84us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:56:34.855649 (+ 15965us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35179/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 04:56:34.856073 (+ 424us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.243.254:44147: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35179/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":343,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.243.254:44147: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35179/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 04:56:34.859798 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 6623
I20260809 04:56:34.861032 6719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:34.955644 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 6623
2026-08-09T04:56:35Z chronyd exiting
I20260809 04:56:35.017052 30671 test_util.cc:175] -----------------------------------------------
I20260809 04:56:35.017185 30671 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786251347713215-30671-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (767 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 04:56:35.022161 30671 mini_oidc.cc:152] Starting JWKS server
I20260809 04:56:35.024296 30671 webserver.cc:460] Webserver started at https://127.0.0.1:42651/ using document root <none> and password file <none>
I20260809 04:56:35.024538 30671 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:56:35.025218 30671 webserver.cc:460] Webserver started at http://127.0.0.1:34967/ using document root <none> and password file <none>
2026-08-09T04:56:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:35Z Disabled control of system clock
I20260809 04:56:35.059304 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:33003
--webserver_interface=127.29.243.254
--webserver_port=0
--builtin_ntp_servers=127.29.243.212:38839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:33003
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42651/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/testchainca.cert with env {}
W20260809 04:56:35.390012 6779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:35.390324 6779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:35.390477 6779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:35.397653 6779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:35.397770 6779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:35.397845 6779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:35.397918 6779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:35.397989 6779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:56:35.398053 6779 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:56:35.398093 6779 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42651/jwks.json
I20260809 04:56:35.408087 6779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:33003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:33003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42651/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:35.409443 6779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:35.410759 6779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:35.422432 6786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:35.422464 6789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:35.422741 6785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:35.423321 6779 server_base.cc:1034] running on GCE node
I20260809 04:56:35.424114 6779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:35.425525 6779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:35.426736 6779 hybrid_clock.cc:648] HybridClock initialized: now 1786251395426689 us; error 41 us; skew 500 ppm
I20260809 04:56:35.427090 6779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:56:35.429471 6779 webserver.cc:460] Webserver started at http://127.29.243.254:43853/ using document root <none> and password file <none>
I20260809 04:56:35.430197 6779 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:35.430312 6779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:35.430666 6779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:35.433648 6779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "5cd5f7f908b44279b9a8e467f93809f3"
format_stamp: "Formatted at 2026-08-09 04:56:35 on dist-test-slave-21b9"
server_key: "1dd18865dadc150c349bb0d209c760a0"
server_key_iv: "707093e4c1d17b249112f6fe08c92bfc"
server_key_version: "encryptionkey@0"
I20260809 04:56:35.434442 6779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "5cd5f7f908b44279b9a8e467f93809f3"
format_stamp: "Formatted at 2026-08-09 04:56:35 on dist-test-slave-21b9"
server_key: "1dd18865dadc150c349bb0d209c760a0"
server_key_iv: "707093e4c1d17b249112f6fe08c92bfc"
server_key_version: "encryptionkey@0"
I20260809 04:56:35.439797 6779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 04:56:35.443436 6797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:35.444878 6779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:56:35.445057 6779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "5cd5f7f908b44279b9a8e467f93809f3"
format_stamp: "Formatted at 2026-08-09 04:56:35 on dist-test-slave-21b9"
server_key: "1dd18865dadc150c349bb0d209c760a0"
server_key_iv: "707093e4c1d17b249112f6fe08c92bfc"
server_key_version: "encryptionkey@0"
I20260809 04:56:35.445263 6779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:35.482306 6779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:35.483243 6779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:35.483592 6779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:35.501969 6779 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:33003
I20260809 04:56:35.501945 6875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:33003 every 8 connection(s)
I20260809 04:56:35.503346 6779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:35.507160 6876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:35.510138 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 6779
I20260809 04:56:35.510357 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:35.511164 30671 external_mini_cluster.cc:1428] Setting key 37fba24ff0f63f261eb19af823ed4a8a
I20260809 04:56:35.519979 6876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3: Bootstrap starting.
I20260809 04:56:35.523631 6876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:35.524791 6876 log.cc:824] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:35.525405 6880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:35.513259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40078 (local address 127.29.243.254:33003)
0809 04:56:35.514291 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:35.514302 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:35.514351 (+ 49us) server_negotiation.cc:407] Connection header received
0809 04:56:35.516139 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:35.516176 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:35.516691 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:35.517052 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:35.517630 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:35.519008 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:35.520764 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:35.521488 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:35.521639 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:35.522912 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:35.522921 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:35.522935 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:35.523522 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:35.523606 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:35.523630 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:35.523745 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:35.524025 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 04:56:35.524360 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":353,"threads_started":1}
I20260809 04:56:35.527988 6876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3: No bootstrap required, opened a new log
I20260809 04:56:35.529342 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:56:35.536080 6876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd5f7f908b44279b9a8e467f93809f3" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33003 } }
I20260809 04:56:35.536558 6876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:35.536674 6876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd5f7f908b44279b9a8e467f93809f3, State: Initialized, Role: FOLLOWER
I20260809 04:56:35.537348 6876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd5f7f908b44279b9a8e467f93809f3" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33003 } }
I20260809 04:56:35.537647 6876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:35.537809 6876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:35.537999 6876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:35.541288 6876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd5f7f908b44279b9a8e467f93809f3" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33003 } }
I20260809 04:56:35.541985 6876 leader_election.cc:302] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cd5f7f908b44279b9a8e467f93809f3; no voters:
I20260809 04:56:35.545619 6876 leader_election.cc:288] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:35.545717 6882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:35.547005 6882 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5cd5f7f908b44279b9a8e467f93809f3, State: Running, Role: LEADER
I20260809 04:56:35.547631 6882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd5f7f908b44279b9a8e467f93809f3" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33003 } }
I20260809 04:56:35.548081 6876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:35.550477 6883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cd5f7f908b44279b9a8e467f93809f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd5f7f908b44279b9a8e467f93809f3" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33003 } } }
I20260809 04:56:35.550908 6883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:35.551347 6884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cd5f7f908b44279b9a8e467f93809f3. Latest consensus state: current_term: 1 leader_uuid: "5cd5f7f908b44279b9a8e467f93809f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd5f7f908b44279b9a8e467f93809f3" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33003 } } }
I20260809 04:56:35.551815 6884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:35.552843 6889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:35.558513 6889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:35.567616 6889 catalog_manager.cc:1347] Generated new cluster ID: 8e853e32391c46ddbe0639663aa03e9e
I20260809 04:56:35.567730 6889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:35.591070 6889 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:35.592347 6889 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:35.605652 6889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cd5f7f908b44279b9a8e467f93809f3: Generated new TSK 0
I20260809 04:56:35.606473 6889 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 04:56:35.639021 6919 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 04:56:35.639159 6919 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 04:56:35.640364 6880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:35.619534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40088 (local address 127.29.243.254:33003)
0809 04:56:35.619728 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:35.619738 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:35.620083 (+ 345us) server_negotiation.cc:407] Connection header received
0809 04:56:35.620295 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:35.620305 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:35.620396 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:35.620556 (+ 160us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:56:35.620940 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:35.622072 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:35.623441 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:35.624148 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:35.624284 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:35.624736 (+ 452us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:56:35.639657 (+ 14921us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42651/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 04:56:35.640073 (+ 416us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40088: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42651/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 04:56:35.640517 6880 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40088: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42651/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 04:56:35.640558 6918 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:56:35.619381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33003 (local address 127.0.0.1:40088)
0809 04:56:35.619975 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:35.620002 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:35.620122 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:35.620557 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:35.620560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:35.620569 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:56:35.620805 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:35.620812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:35.622207 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:35.622211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:35.623305 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:35.623320 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:35.624307 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:35.624311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:35.624440 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:35.624472 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:56:35.639906 (+ 15434us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42651/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 04:56:35.640167 (+ 261us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.243.254:33003: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42651/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":329,"threads_started":1}
I20260809 04:56:35.641539 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 6779
I20260809 04:56:35.642769 6869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:35.735894 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 6779
2026-08-09T04:56:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (775 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 04:56:35.796655 30671 mini_oidc.cc:152] Starting JWKS server
I20260809 04:56:35.798599 30671 webserver.cc:460] Webserver started at https://127.0.0.1:43565/ using document root <none> and password file <none>
I20260809 04:56:35.798837 30671 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:56:35.799539 30671 webserver.cc:460] Webserver started at http://127.0.0.1:33459/ using document root <none> and password file <none>
2026-08-09T04:56:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:35Z Disabled control of system clock
I20260809 04:56:35.834635 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35551
--webserver_interface=127.29.243.254
--webserver_port=0
--builtin_ntp_servers=127.29.243.212:37699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:35551
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43565/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 04:56:36.172909 6933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:36.173274 6933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:36.173414 6933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:36.180756 6933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:36.180881 6933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:36.180974 6933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:36.181057 6933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:36.181118 6933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:56:36.181205 6933 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:56:36.181259 6933 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43565/jwks.json
I20260809 04:56:36.190466 6933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:37699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:35551
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43565/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:36.191862 6933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:36.193861 6933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:36.205631 6942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:36.205586 6944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:36.205636 6946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:36.206931 6933 server_base.cc:1034] running on GCE node
I20260809 04:56:36.207758 6933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:36.209165 6933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:36.210374 6933 hybrid_clock.cc:648] HybridClock initialized: now 1786251396210322 us; error 44 us; skew 500 ppm
I20260809 04:56:36.210738 6933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:56:36.217365 6933 webserver.cc:460] Webserver started at http://127.29.243.254:43207/ using document root <none> and password file <none>
I20260809 04:56:36.218137 6933 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:36.218246 6933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:36.218578 6933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:36.221616 6933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "a29df29ac90340d788fb0118cb9ea66b"
format_stamp: "Formatted at 2026-08-09 04:56:36 on dist-test-slave-21b9"
server_key: "33a856a24a830bcf8f2e9c668685a548"
server_key_iv: "80155f2beef43f235cc9adeb2c6deb7d"
server_key_version: "encryptionkey@0"
I20260809 04:56:36.222406 6933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "a29df29ac90340d788fb0118cb9ea66b"
format_stamp: "Formatted at 2026-08-09 04:56:36 on dist-test-slave-21b9"
server_key: "33a856a24a830bcf8f2e9c668685a548"
server_key_iv: "80155f2beef43f235cc9adeb2c6deb7d"
server_key_version: "encryptionkey@0"
I20260809 04:56:36.227639 6933 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 04:56:36.231046 6954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:36.232425 6933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:36.232604 6933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "a29df29ac90340d788fb0118cb9ea66b"
format_stamp: "Formatted at 2026-08-09 04:56:36 on dist-test-slave-21b9"
server_key: "33a856a24a830bcf8f2e9c668685a548"
server_key_iv: "80155f2beef43f235cc9adeb2c6deb7d"
server_key_version: "encryptionkey@0"
I20260809 04:56:36.232810 6933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:36.249667 6933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:36.250527 6933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:36.250826 6933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:36.269256 6933 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:35551
I20260809 04:56:36.269294 7025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:35551 every 8 connection(s)
I20260809 04:56:36.270622 6933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:36.274595 7028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:36.275873 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 6933
I20260809 04:56:36.276096 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:36.276911 30671 external_mini_cluster.cc:1428] Setting key 19827c8860a921e5a504b64cacaf8f62
I20260809 04:56:36.287815 7028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b: Bootstrap starting.
I20260809 04:56:36.290510 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:36.279138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49472 (local address 127.29.243.254:35551)
0809 04:56:36.280344 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:36.280356 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:36.280398 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:56:36.282051 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:36.282079 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:36.282565 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:36.282930 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:36.283549 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:36.284889 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:36.286217 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:36.286890 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:36.287058 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:36.288234 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:36.288243 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:36.288256 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:36.288785 (+ 529us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:36.288864 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:36.288885 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:36.288987 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:36.289253 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 04:56:36.289571 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":418,"threads_started":1}
I20260809 04:56:36.291116 7028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:36.292337 7028 log.cc:824] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:36.294613 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:56:36.295504 7028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b: No bootstrap required, opened a new log
I20260809 04:56:36.303540 7028 raft_consensus.cc:348] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29df29ac90340d788fb0118cb9ea66b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35551 } }
I20260809 04:56:36.303886 7028 raft_consensus.cc:374] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:36.303989 7028 raft_consensus.cc:729] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a29df29ac90340d788fb0118cb9ea66b, State: Initialized, Role: FOLLOWER
I20260809 04:56:36.304590 7028 consensus_queue.cc:260] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29df29ac90340d788fb0118cb9ea66b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35551 } }
I20260809 04:56:36.304807 7028 raft_consensus.cc:388] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:36.304934 7028 raft_consensus.cc:482] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:36.305100 7028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:36.307535 7028 raft_consensus.cc:504] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29df29ac90340d788fb0118cb9ea66b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35551 } }
I20260809 04:56:36.308069 7028 leader_election.cc:302] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a29df29ac90340d788fb0118cb9ea66b; no voters:
I20260809 04:56:36.308645 7028 leader_election.cc:288] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:36.308816 7037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:36.309525 7037 raft_consensus.cc:686] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [term 1 LEADER]: Becoming Leader. State: Replica: a29df29ac90340d788fb0118cb9ea66b, State: Running, Role: LEADER
I20260809 04:56:36.310012 7037 consensus_queue.cc:237] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29df29ac90340d788fb0118cb9ea66b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35551 } }
I20260809 04:56:36.310702 7028 sys_catalog.cc:564] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:36.312757 7040 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a29df29ac90340d788fb0118cb9ea66b. Latest consensus state: current_term: 1 leader_uuid: "a29df29ac90340d788fb0118cb9ea66b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29df29ac90340d788fb0118cb9ea66b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35551 } } }
I20260809 04:56:36.313186 7040 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:36.312963 7038 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a29df29ac90340d788fb0118cb9ea66b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29df29ac90340d788fb0118cb9ea66b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35551 } } }
I20260809 04:56:36.313674 7038 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:36.315464 7053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:36.319808 7053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:36.329419 7053 catalog_manager.cc:1347] Generated new cluster ID: 21353f9a140a4e4e9f0d9cd0281894c7
I20260809 04:56:36.329537 7053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:36.339181 7053 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:36.340489 7053 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:36.348537 7053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a29df29ac90340d788fb0118cb9ea66b: Generated new TSK 0
I20260809 04:56:36.349577 7053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:56:36.376159 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:36.368397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49482 (local address 127.29.243.254:35551)
0809 04:56:36.368624 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:36.368631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:36.368944 (+ 313us) server_negotiation.cc:407] Connection header received
0809 04:56:36.369240 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:36.369244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:36.369317 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:36.369494 (+ 177us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:56:36.369983 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:36.370714 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:36.372131 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:36.372595 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:36.372708 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:36.373255 (+ 547us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:56:36.375512 (+ 2257us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43565/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 04:56:36.375841 (+ 329us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49482: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43565/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 04:56:36.376291 7033 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49482: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43565/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 04:56:36.378283 7077 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:56:36.368237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35551 (local address 127.0.0.1:49482)
0809 04:56:36.368829 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:36.368849 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:36.369030 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:36.369494 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:36.369499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:36.369513 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:56:36.369820 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:36.369831 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:36.370862 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:36.370867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:36.371981 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:36.371995 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:36.372795 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:36.372802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:36.372927 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:36.372972 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:56:36.375728 (+ 2756us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43565/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 04:56:36.377838 (+ 2110us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.243.254:35551: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43565/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":1779,"thread_start_us":404,"threads_started":1}
I20260809 04:56:36.379675 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 6933
I20260809 04:56:36.380918 7020 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 04:56:36.390925 7078 webserver.cc:533] Webserver: sslize failed:
E20260809 04:56:36.391074 7078 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 04:56:36.473116 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 6933
2026-08-09T04:56:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (740 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T04:56:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:36Z Disabled control of system clock
I20260809 04:56:36.569895 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41833
--webserver_interface=127.29.243.254
--webserver_port=0
--builtin_ntp_servers=127.29.243.212:35739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41833
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/testchainca.cert with env {}
W20260809 04:56:36.902050 7084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:36.902364 7084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:36.902540 7084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:36.909834 7084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:36.909940 7084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:36.910004 7084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:36.910091 7084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:36.910159 7084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:56:36.910247 7084 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:56:36.910306 7084 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 04:56:36.920315 7084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:35739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41833
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:36.921594 7084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:36.922857 7084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:36.934713 7094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:36.934785 7089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:36.934800 7091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:36.935678 7084 server_base.cc:1034] running on GCE node
I20260809 04:56:36.936431 7084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:36.937783 7084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:36.938984 7084 hybrid_clock.cc:648] HybridClock initialized: now 1786251396938939 us; error 38 us; skew 500 ppm
I20260809 04:56:36.939347 7084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:56:36.941557 7084 webserver.cc:460] Webserver started at http://127.29.243.254:44907/ using document root <none> and password file <none>
I20260809 04:56:36.942214 7084 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:36.942304 7084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:36.942582 7084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:36.945430 7084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "2911c98ec7d440f1b4e6a3c7ce6a7ab7"
format_stamp: "Formatted at 2026-08-09 04:56:36 on dist-test-slave-21b9"
server_key: "5fa219a05f425b4163a884695742e289"
server_key_iv: "8bb7362eaf78364f8936202ffe8db77c"
server_key_version: "encryptionkey@0"
I20260809 04:56:36.946182 7084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "2911c98ec7d440f1b4e6a3c7ce6a7ab7"
format_stamp: "Formatted at 2026-08-09 04:56:36 on dist-test-slave-21b9"
server_key: "5fa219a05f425b4163a884695742e289"
server_key_iv: "8bb7362eaf78364f8936202ffe8db77c"
server_key_version: "encryptionkey@0"
I20260809 04:56:36.951256 7084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 04:56:36.954604 7101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:36.955909 7084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 04:56:36.956058 7084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "2911c98ec7d440f1b4e6a3c7ce6a7ab7"
format_stamp: "Formatted at 2026-08-09 04:56:36 on dist-test-slave-21b9"
server_key: "5fa219a05f425b4163a884695742e289"
server_key_iv: "8bb7362eaf78364f8936202ffe8db77c"
server_key_version: "encryptionkey@0"
I20260809 04:56:36.956231 7084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:36.971998 7084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:36.972767 7084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:36.973001 7084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:36.990989 7084 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:41833
I20260809 04:56:36.991019 7193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:41833 every 8 connection(s)
I20260809 04:56:36.992336 7084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:36.996295 7194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:36.999914 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 7084
I20260809 04:56:37.000154 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:37.000964 30671 external_mini_cluster.cc:1428] Setting key 7588338a7568716b4982ae437d68c8a3
I20260809 04:56:37.009168 7194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7: Bootstrap starting.
I20260809 04:56:37.012372 7194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:37.013473 7194 log.cc:824] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:37.014360 7198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:37.002894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32854 (local address 127.29.243.254:41833)
0809 04:56:37.003967 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:37.003978 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:37.004022 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:56:37.005639 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:37.005664 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:37.006187 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:37.006571 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:37.007167 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:37.008514 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:37.009840 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:37.010532 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:37.010724 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:37.011892 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:37.011900 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:37.011915 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:56:37.012455 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:37.012576 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:56:37.012604 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:37.012718 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:37.012929 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 04:56:37.013242 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":394,"threads_started":1}
I20260809 04:56:37.016702 7194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7: No bootstrap required, opened a new log
I20260809 04:56:37.018164 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:56:37.024267 7194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2911c98ec7d440f1b4e6a3c7ce6a7ab7" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41833 } }
I20260809 04:56:37.024684 7194 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:37.024788 7194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2911c98ec7d440f1b4e6a3c7ce6a7ab7, State: Initialized, Role: FOLLOWER
I20260809 04:56:37.025315 7194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2911c98ec7d440f1b4e6a3c7ce6a7ab7" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41833 } }
I20260809 04:56:37.025545 7194 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:37.025668 7194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:37.025830 7194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:37.028169 7194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2911c98ec7d440f1b4e6a3c7ce6a7ab7" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41833 } }
I20260809 04:56:37.028678 7194 leader_election.cc:302] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2911c98ec7d440f1b4e6a3c7ce6a7ab7; no voters:
I20260809 04:56:37.029264 7194 leader_election.cc:288] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:37.029415 7200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:37.030119 7200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [term 1 LEADER]: Becoming Leader. State: Replica: 2911c98ec7d440f1b4e6a3c7ce6a7ab7, State: Running, Role: LEADER
I20260809 04:56:37.030654 7200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2911c98ec7d440f1b4e6a3c7ce6a7ab7" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41833 } }
I20260809 04:56:37.031286 7194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:37.033298 7202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2911c98ec7d440f1b4e6a3c7ce6a7ab7. Latest consensus state: current_term: 1 leader_uuid: "2911c98ec7d440f1b4e6a3c7ce6a7ab7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2911c98ec7d440f1b4e6a3c7ce6a7ab7" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41833 } } }
I20260809 04:56:37.033654 7202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:37.033516 7201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2911c98ec7d440f1b4e6a3c7ce6a7ab7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2911c98ec7d440f1b4e6a3c7ce6a7ab7" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41833 } } }
I20260809 04:56:37.033788 7201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:37.035233 7212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:37.040599 7212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:37.049270 7212 catalog_manager.cc:1347] Generated new cluster ID: 834f3b29b5dd4b9f806eef556a7310f7
I20260809 04:56:37.049399 7212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:37.070762 7212 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:37.072664 7212 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:37.091454 7212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2911c98ec7d440f1b4e6a3c7ce6a7ab7: Generated new TSK 0
I20260809 04:56:37.092350 7212 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:56:37.169790 7198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:37.107582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.29.243.254:41833)
0809 04:56:37.107749 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:37.107756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:37.108153 (+ 397us) server_negotiation.cc:407] Connection header received
0809 04:56:37.108373 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:37.108377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:37.108455 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:37.108584 (+ 129us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:56:37.109044 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:37.109769 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:37.111325 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:37.111782 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:37.111896 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:37.112372 (+ 476us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:56:37.169004 (+ 56632us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 04:56:37.169461 (+ 457us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32860: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 04:56:37.169939 7198 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32860: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 04:56:37.170033 7233 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:56:37.107508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41833 (local address 127.0.0.1:32860)
0809 04:56:37.108048 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:37.108064 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:37.108202 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:37.108617 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:37.108622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:37.108634 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:56:37.108894 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:37.108903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:37.109908 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:37.109912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:37.111163 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:37.111177 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:37.111951 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:37.111958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:37.112086 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:37.112118 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:56:37.169337 (+ 57219us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 04:56:37.169658 (+ 321us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.243.254:41833: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":283,"threads_started":1}
I20260809 04:56:37.171880 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 7084
I20260809 04:56:37.173108 7184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:37.272469 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 7084
2026-08-09T04:56:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (781 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (426 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (428 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:56:38 dist-test-slave-21b9 krb5kdc[7258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:56:38 dist-test-slave-21b9 krb5kdc[7258](info): set up 2 sockets
Aug 09 04:56:38 dist-test-slave-21b9 krb5kdc[7258](info): commencing operation
krb5kdc: starting...
W20260809 04:56:39.348985 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:56:39 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:56:39.584290 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:46169
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:46169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:39.919130 7281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:39.919456 7281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:39.919607 7281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:39.926764 7281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:39.926887 7281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:39.926970 7281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:39.927057 7281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:39.927121 7281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:39.936966 7281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:46169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:46169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:39.938341 7281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:39.939674 7281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:39.951304 7287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:39.951334 7291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:39.951498 7293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:39.952126 7281 server_base.cc:1034] running on GCE node
I20260809 04:56:39.952873 7281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:39.954262 7281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:39.955469 7281 hybrid_clock.cc:648] HybridClock initialized: now 1786251399955406 us; error 43 us; skew 500 ppm
I20260809 04:56:39.955812 7281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:39 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251399, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:39.959048 7281 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:56:39.960510 7281 webserver.cc:460] Webserver started at http://127.29.243.254:42643/ using document root <none> and password file <none>
I20260809 04:56:39.961174 7281 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:39.961277 7281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:39.961549 7281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:39.964540 7281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "89f4122c824f4393ab3a692cec0a3dca"
format_stamp: "Formatted at 2026-08-09 04:56:39 on dist-test-slave-21b9"
server_key: "90c487650dafdc93601c9769286291d5"
server_key_iv: "d50b2e815258b79444768f77fefa5d5f"
server_key_version: "encryptionkey@0"
I20260809 04:56:39.965271 7281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "89f4122c824f4393ab3a692cec0a3dca"
format_stamp: "Formatted at 2026-08-09 04:56:39 on dist-test-slave-21b9"
server_key: "90c487650dafdc93601c9769286291d5"
server_key_iv: "d50b2e815258b79444768f77fefa5d5f"
server_key_version: "encryptionkey@0"
I20260809 04:56:39.970379 7281 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 04:56:39.973863 7304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:39.975147 7281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:39.975337 7281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "89f4122c824f4393ab3a692cec0a3dca"
format_stamp: "Formatted at 2026-08-09 04:56:39 on dist-test-slave-21b9"
server_key: "90c487650dafdc93601c9769286291d5"
server_key_iv: "d50b2e815258b79444768f77fefa5d5f"
server_key_version: "encryptionkey@0"
I20260809 04:56:39.975524 7281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:39.992417 7281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:39.996228 7281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:39.996492 7281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:40.022087 7281 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:46169
I20260809 04:56:40.022120 7390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:46169 every 8 connection(s)
I20260809 04:56:40.023586 7281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:40.027025 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 7281
I20260809 04:56:40.027287 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:40.027768 7391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:40.028090 30671 external_mini_cluster.cc:1428] Setting key baeead4f2785f6b94a36bd430248bbff
I20260809 04:56:40.042363 7391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca: Bootstrap starting.
Aug 09 04:56:40 dist-test-slave-21b9 krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:40.045594 7391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:40.046639 7391 log.cc:824] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:40.049443 7391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca: No bootstrap required, opened a new log
I20260809 04:56:40.050753 7395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:40.030182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53588 (local address 127.29.243.254:46169)
0809 04:56:40.031006 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:40.031021 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:40.031067 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:56:40.032655 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:40.032678 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:40.033070 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:40.033399 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:40.035184 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:40.036538 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:40.038260 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:40.038963 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:40.039112 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:40.043811 (+ 4699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:40.043819 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:40.043834 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:40.043875 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:40.046894 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:40.047594 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:40.047602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:40.047609 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:40.047778 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:40.048145 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:40.048149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:40.048151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:40.048666 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:40.048951 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:40.049386 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 04:56:40.049795 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":372,"threads_started":1}
I20260809 04:56:40.057226 7391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f4122c824f4393ab3a692cec0a3dca" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46169 } }
I20260809 04:56:40.057600 7391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:40.057763 7391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f4122c824f4393ab3a692cec0a3dca, State: Initialized, Role: FOLLOWER
I20260809 04:56:40.058300 7391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f4122c824f4393ab3a692cec0a3dca" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46169 } }
I20260809 04:56:40.058530 7391 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:40.058658 7391 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:40.058813 7391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:40.060936 7391 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f4122c824f4393ab3a692cec0a3dca" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46169 } }
I20260809 04:56:40.061472 7391 leader_election.cc:302] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89f4122c824f4393ab3a692cec0a3dca; no voters:
I20260809 04:56:40.062079 7391 leader_election.cc:288] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:40.062198 7398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:40.062865 7398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [term 1 LEADER]: Becoming Leader. State: Replica: 89f4122c824f4393ab3a692cec0a3dca, State: Running, Role: LEADER
I20260809 04:56:40.063416 7398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f4122c824f4393ab3a692cec0a3dca" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46169 } }
I20260809 04:56:40.064138 7391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:40.066177 7400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89f4122c824f4393ab3a692cec0a3dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f4122c824f4393ab3a692cec0a3dca" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46169 } } }
I20260809 04:56:40.066561 7400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:40.066998 7404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89f4122c824f4393ab3a692cec0a3dca. Latest consensus state: current_term: 1 leader_uuid: "89f4122c824f4393ab3a692cec0a3dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f4122c824f4393ab3a692cec0a3dca" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 46169 } } }
I20260809 04:56:40.067299 7404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:40.068444 7411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:40.073556 7411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:40.081529 7411 catalog_manager.cc:1347] Generated new cluster ID: 73459462fc77410fa5ae022ec69eb91b
I20260809 04:56:40.081640 7411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:40.090433 7411 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:40.092329 7411 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:40.105526 7411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89f4122c824f4393ab3a692cec0a3dca: Generated new TSK 0
I20260809 04:56:40.106613 7411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:56:40.165354 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:46169
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:40.503970 7436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:40.504266 7436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:40.504422 7436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:40.511511 7436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:40.511631 7436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:40.511780 7436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:56:40.521222 7436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:46169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:40.522528 7436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:40.523876 7436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:40.537123 7446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:40.537169 7447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:40.537213 7449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:40.538239 7436 server_base.cc:1034] running on GCE node
I20260809 04:56:40.538818 7436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:40.539796 7436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:40.541007 7436 hybrid_clock.cc:648] HybridClock initialized: now 1786251400540948 us; error 44 us; skew 500 ppm
I20260809 04:56:40.541304 7436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:40 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251400, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:40.544407 7436 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:56:40.545816 7436 webserver.cc:460] Webserver started at http://127.29.243.193:41399/ using document root <none> and password file <none>
I20260809 04:56:40.546483 7436 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:40.546607 7436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:40.546907 7436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:40.549966 7436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "4ea6b549c857425985fd30edf25f1575"
format_stamp: "Formatted at 2026-08-09 04:56:40 on dist-test-slave-21b9"
server_key: "c0f01a70b2b62319d5061e4d0a884d73"
server_key_iv: "f1c905ef00ecd1634379c926b63d1eaa"
server_key_version: "encryptionkey@0"
I20260809 04:56:40.550743 7436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "4ea6b549c857425985fd30edf25f1575"
format_stamp: "Formatted at 2026-08-09 04:56:40 on dist-test-slave-21b9"
server_key: "c0f01a70b2b62319d5061e4d0a884d73"
server_key_iv: "f1c905ef00ecd1634379c926b63d1eaa"
server_key_version: "encryptionkey@0"
I20260809 04:56:40.557652 7436 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260809 04:56:40.562291 7461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:40.564024 7436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:40.564191 7436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "4ea6b549c857425985fd30edf25f1575"
format_stamp: "Formatted at 2026-08-09 04:56:40 on dist-test-slave-21b9"
server_key: "c0f01a70b2b62319d5061e4d0a884d73"
server_key_iv: "f1c905ef00ecd1634379c926b63d1eaa"
server_key_version: "encryptionkey@0"
I20260809 04:56:40.564375 7436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:40.597090 7436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:40.601303 7436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:40.601565 7436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:40.602576 7436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:40.603930 7436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:40.604004 7436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:40.604128 7436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:40.604187 7436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:40.635443 7436 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:41339
I20260809 04:56:40.635540 7633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:41339 every 8 connection(s)
I20260809 04:56:40.636744 7436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:56:40.641589 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 7436
I20260809 04:56:40.641812 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:56:40.642633 30671 external_mini_cluster.cc:1428] Setting key eada305a989c0933ff2c346720a26759
Aug 09 04:56:40 dist-test-slave-21b9 krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251400, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:40.657171 7636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:40.639330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:60397 (local address 127.29.243.254:46169)
0809 04:56:40.639776 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:40.639780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:40.640772 (+ 992us) server_negotiation.cc:407] Connection header received
0809 04:56:40.643050 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:40.643055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:40.643139 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:40.643235 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:40.645410 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:40.646126 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:40.647698 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:40.648103 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:40.648217 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:40.651694 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:40.651699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:40.651703 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:40.651736 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:40.654130 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:40.654850 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:40.654855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:40.654859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:40.655010 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:40.655447 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:40.655452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:40.655455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:40.655786 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:40.655933 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:40.656600 (+ 667us) server_negotiation.cc:299] Negotiation successful
0809 04:56:40.656825 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":214,"threads_started":1}
I20260809 04:56:40.658041 7637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:40.639901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:46169 (local address 127.29.243.193:60397)
0809 04:56:40.640618 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:40.640664 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:40.642742 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:40.643833 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:40.643845 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:40.644644 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:40.645241 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:40.645260 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:40.646275 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:40.646281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:40.647565 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:40.647578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:40.648327 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:40.648334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:40.648559 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:40.649256 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:40.649287 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:40.651453 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:40.654275 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:40.654282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:40.654296 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:40.654712 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:40.655113 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:40.655117 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:40.655119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:40.655329 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:40.655891 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:40.655897 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:40.656215 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 04:56:40.656453 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 04:56:40.656828 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":273,"threads_started":1}
I20260809 04:56:40.659663 7634 heartbeater.cc:347] Connected to a master server at 127.29.243.254:46169
I20260809 04:56:40.659967 7634 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:40.660694 7634 heartbeater.cc:511] Master 127.29.243.254:46169 requested a full tablet report, sending...
I20260809 04:56:40.663069 7335 ts_manager.cc:194] Registered new tserver with Master: 4ea6b549c857425985fd30edf25f1575 (127.29.243.193:41339)
I20260809 04:56:40.665228 7335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:60397
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:56:40.699146 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:46169
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:41.032483 7641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:41.032757 7641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:41.032882 7641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:41.040119 7641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:41.040227 7641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:41.040375 7641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:56:41.050138 7641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:46169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:41.051560 7641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:41.052982 7641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:41.066262 7649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:41.066310 7651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:41.066469 7653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:41.067471 7641 server_base.cc:1034] running on GCE node
I20260809 04:56:41.068055 7641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:41.069103 7641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:41.070319 7641 hybrid_clock.cc:648] HybridClock initialized: now 1786251401070249 us; error 55 us; skew 500 ppm
I20260809 04:56:41.070678 7641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:41.073762 7641 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:56:41.075264 7641 webserver.cc:460] Webserver started at http://127.29.243.194:38329/ using document root <none> and password file <none>
I20260809 04:56:41.075984 7641 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:41.076090 7641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:41.076365 7641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:41.079466 7641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "92523ecd880148c78818b39d50fff799"
format_stamp: "Formatted at 2026-08-09 04:56:41 on dist-test-slave-21b9"
server_key: "600ee9bfaeec103fbbc1d123717d61dc"
server_key_iv: "85d2211d027ce20086b26bade504d396"
server_key_version: "encryptionkey@0"
I20260809 04:56:41.080232 7641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "92523ecd880148c78818b39d50fff799"
format_stamp: "Formatted at 2026-08-09 04:56:41 on dist-test-slave-21b9"
server_key: "600ee9bfaeec103fbbc1d123717d61dc"
server_key_iv: "85d2211d027ce20086b26bade504d396"
server_key_version: "encryptionkey@0"
I20260809 04:56:41.085299 7641 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:56:41.088824 7662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:41.090178 7641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:56:41.090337 7641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "92523ecd880148c78818b39d50fff799"
format_stamp: "Formatted at 2026-08-09 04:56:41 on dist-test-slave-21b9"
server_key: "600ee9bfaeec103fbbc1d123717d61dc"
server_key_iv: "85d2211d027ce20086b26bade504d396"
server_key_version: "encryptionkey@0"
I20260809 04:56:41.090539 7641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:41.113628 7641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:41.117587 7641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:41.117865 7641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:41.118839 7641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:41.120223 7641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:41.120301 7641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:41.120409 7641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:41.120486 7641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:41.151240 7641 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:35659
I20260809 04:56:41.151351 7831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:35659 every 8 connection(s)
I20260809 04:56:41.152613 7641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:56:41.161194 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 7641
I20260809 04:56:41.161389 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:56:41.161782 30671 external_mini_cluster.cc:1428] Setting key 4a24c39584c63a1591ebfb095b574bf6
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:41.173324 7636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:41.155150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:52055 (local address 127.29.243.254:46169)
0809 04:56:41.155389 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:41.155396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:41.156600 (+ 1204us) server_negotiation.cc:407] Connection header received
0809 04:56:41.158377 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:41.158382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:41.158506 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:41.158626 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:41.160175 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.161002 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.162701 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.163200 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.163350 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:41.166878 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:41.166884 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:41.166888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:41.166928 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:41.170167 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.170937 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.170942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.170945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.171099 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.171590 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.171595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.171597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.171923 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:41.172063 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:41.172796 (+ 733us) server_negotiation.cc:299] Negotiation successful
0809 04:56:41.173015 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:56:41.174328 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:41.155671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:46169 (local address 127.29.243.194:52055)
0809 04:56:41.156412 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:41.156484 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:41.158094 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:41.158836 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:41.158844 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:41.159423 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:41.159993 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:41.160012 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.161179 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.161183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:41.162541 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:41.162557 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.163459 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.163468 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:41.163758 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:41.164475 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:41.164499 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:41.166642 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:41.170297 (+ 3655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:41.170304 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:41.170318 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:41.170814 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:41.171208 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:41.171212 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:41.171214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:41.171475 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:41.172078 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:41.172085 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:41.172416 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 04:56:41.172668 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 04:56:41.173083 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":280,"threads_started":1}
I20260809 04:56:41.175859 7832 heartbeater.cc:347] Connected to a master server at 127.29.243.254:46169
I20260809 04:56:41.176173 7832 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:41.176882 7832 heartbeater.cc:511] Master 127.29.243.254:46169 requested a full tablet report, sending...
I20260809 04:56:41.178375 7335 ts_manager.cc:194] Registered new tserver with Master: 92523ecd880148c78818b39d50fff799 (127.29.243.194:35659)
I20260809 04:56:41.179261 7335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:52055
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:56:41.216728 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:46169
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:41.550195 7839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:41.550495 7839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:41.550640 7839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:41.557818 7839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:41.557935 7839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:41.558090 7839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:56:41.567690 7839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:46169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:41.569046 7839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:41.570500 7839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:41.583154 7847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:41.583190 7848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:41.583112 7850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:41.584254 7839 server_base.cc:1034] running on GCE node
I20260809 04:56:41.584842 7839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:41.585773 7839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:41.586979 7839 hybrid_clock.cc:648] HybridClock initialized: now 1786251401586927 us; error 41 us; skew 500 ppm
I20260809 04:56:41.587324 7839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:41.594481 7839 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:56:41.595958 7839 webserver.cc:460] Webserver started at http://127.29.243.195:39247/ using document root <none> and password file <none>
I20260809 04:56:41.596643 7839 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:41.596773 7839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:41.597084 7839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:41.600077 7839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "98fa77e02bcf423f9ad70c5ab42741e1"
format_stamp: "Formatted at 2026-08-09 04:56:41 on dist-test-slave-21b9"
server_key: "ba0fc8278da8cec59c53427152fe2407"
server_key_iv: "7c8f9a8add802d0fa0a7e18a8c7aebf2"
server_key_version: "encryptionkey@0"
I20260809 04:56:41.600821 7839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "98fa77e02bcf423f9ad70c5ab42741e1"
format_stamp: "Formatted at 2026-08-09 04:56:41 on dist-test-slave-21b9"
server_key: "ba0fc8278da8cec59c53427152fe2407"
server_key_iv: "7c8f9a8add802d0fa0a7e18a8c7aebf2"
server_key_version: "encryptionkey@0"
I20260809 04:56:41.605841 7839 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:56:41.609186 7859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:41.610493 7839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:41.610661 7839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "98fa77e02bcf423f9ad70c5ab42741e1"
format_stamp: "Formatted at 2026-08-09 04:56:41 on dist-test-slave-21b9"
server_key: "ba0fc8278da8cec59c53427152fe2407"
server_key_iv: "7c8f9a8add802d0fa0a7e18a8c7aebf2"
server_key_version: "encryptionkey@0"
I20260809 04:56:41.610843 7839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:41.633764 7839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:41.639094 7839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:41.639411 7839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:41.640551 7839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:41.642084 7839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:41.642159 7839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:41.642249 7839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:41.642292 7839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:41.668584 7634 heartbeater.cc:503] Master 127.29.243.254:46169 was elected leader, sending a full tablet report...
I20260809 04:56:41.673043 7839 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:32823
I20260809 04:56:41.673105 8047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:32823 every 8 connection(s)
I20260809 04:56:41.674393 7839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:56:41.680359 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 7839
I20260809 04:56:41.680560 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:56:41.680959 30671 external_mini_cluster.cc:1428] Setting key 9025e20da782e4efb679685b78d40e2d
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:56:41.693859 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:41.676798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:44639 (local address 127.29.243.254:46169)
0809 04:56:41.677239 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:41.677246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:41.678279 (+ 1033us) server_negotiation.cc:407] Connection header received
0809 04:56:41.680146 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:41.680151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:41.680241 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:41.680364 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:41.682066 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.683176 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.685099 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.685518 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.685614 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:41.688883 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:41.688889 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:41.688894 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:41.688933 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:41.691307 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.691964 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.691967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.691970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.692091 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.692457 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.692460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.692462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.692705 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:41.692818 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:41.693433 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 04:56:41.693596 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":198,"threads_started":1}
I20260809 04:56:41.694940 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:41.677379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:46169 (local address 127.29.243.195:44639)
0809 04:56:41.678126 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:41.678166 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:41.679870 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:41.680589 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:41.680599 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:41.681180 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:41.681855 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:41.681884 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.683361 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.683367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:41.684960 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:41.684973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.685695 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.685700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:41.685900 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:41.686503 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:41.686526 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:41.688689 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:41.691423 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:41.691431 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:41.691443 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:41.691853 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:41.692167 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:41.692170 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:41.692172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:41.692346 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:41.692791 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:41.692797 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:41.693099 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 04:56:41.693327 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 04:56:41.693688 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":283,"threads_started":1}
I20260809 04:56:41.696467 8048 heartbeater.cc:347] Connected to a master server at 127.29.243.254:46169
I20260809 04:56:41.696780 8048 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:41.697469 8048 heartbeater.cc:511] Master 127.29.243.254:46169 requested a full tablet report, sending...
I20260809 04:56:41.698814 7335 ts_manager.cc:194] Registered new tserver with Master: 98fa77e02bcf423f9ad70c5ab42741e1 (127.29.243.195:32823)
I20260809 04:56:41.699638 7335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:44639
I20260809 04:56:41.708736 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/test-admin.keytab.
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:41.740463 30671 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 04:56:41.745700 8062 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:56:41.743065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.29.243.193:41339)
0809 04:56:41.743722 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:41.743731 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:41.743777 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:56:41.743848 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:41.743855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:41.744093 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:41.744238 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:41.745289 (+ 1051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40534: BlockingRecv error: recv got EOF from 127.0.0.1:40534 (error 108)
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":310,"threads_started":1}
W20260809 04:56:41.746076 30671 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5cc)
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:41.747584 30671 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:41.749033 30671 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:56:41.763764 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:41.750704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40548 (local address 127.29.243.193:41339)
0809 04:56:41.750888 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:41.750892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:41.751018 (+ 126us) server_negotiation.cc:407] Connection header received
0809 04:56:41.751179 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:41.751183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:41.751243 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:41.751353 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:41.752355 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.753340 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.754537 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.755069 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.755174 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:41.758522 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:41.758536 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:41.758552 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:41.758584 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:41.761294 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.761888 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.761895 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.761897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.762001 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.762324 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.762327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.762329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.762695 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:41.762913 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:41.763171 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 04:56:41.763472 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 04:56:41.769562 8062 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:56:41.767888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.29.243.193:41339)
0809 04:56:41.768127 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:41.768132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:41.768250 (+ 118us) server_negotiation.cc:407] Connection header received
0809 04:56:41.768366 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:41.768369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:41.768423 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:41.768496 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:41.769261 (+ 765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40550: BlockingRecv error: recv got EOF from 127.0.0.1:40550 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 04:56:41.769688 30671 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5cc' of type FILE
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:41.771195 30671 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:41.772491 30671 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:56:41.785645 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:41.773699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.29.243.193:41339)
0809 04:56:41.773852 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:41.773858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:41.774070 (+ 212us) server_negotiation.cc:407] Connection header received
0809 04:56:41.774202 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:41.774205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:41.774267 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:41.774339 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:41.775221 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.775975 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.777128 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.777525 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.777629 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:41.780907 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:41.780912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:41.780915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:41.780954 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:41.783321 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.783905 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.783910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.783914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.784037 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.784399 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.784404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.784408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.784679 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:41.784922 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:41.785192 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 04:56:41.785381 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 04:56:41.789762 8062 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:56:41.788095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40572 (local address 127.29.243.193:41339)
0809 04:56:41.788218 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:41.788222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:41.788661 (+ 439us) server_negotiation.cc:407] Connection header received
0809 04:56:41.788755 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:41.788758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:41.788824 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:41.788908 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:41.789562 (+ 654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40572: BlockingRecv error: recv got EOF from 127.0.0.1:40572 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 04:56:41.789953 30671 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786251347713215-30671-0/krb5kdc/krb5cc' of type FILE
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:41.791347 30671 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:41.792627 30671 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:56:41.805645 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:41.793812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40576 (local address 127.29.243.193:41339)
0809 04:56:41.794011 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:41.794016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:41.794173 (+ 157us) server_negotiation.cc:407] Connection header received
0809 04:56:41.794304 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:41.794308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:41.794399 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:41.794467 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:41.795402 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.796124 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.797301 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.797710 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.797803 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:41.801091 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:41.801096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:41.801099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:41.801130 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:41.803454 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.804057 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.804062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.804064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.804227 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.804521 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.804524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.804526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.804773 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:41.805005 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:41.805164 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 04:56:41.805323 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:56:41.807109 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 7436
I20260809 04:56:41.817967 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:41.807801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.29.243.193:41339)
0809 04:56:41.807939 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:41.807943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:41.808265 (+ 322us) server_negotiation.cc:407] Connection header received
0809 04:56:41.808373 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:41.808376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:41.808428 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:41.808500 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:41.809310 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.809935 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.811120 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.811594 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.811683 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:41.813465 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:41.813470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:41.813473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:41.813500 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:41.815895 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.816466 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.816470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.816472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.816602 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.816877 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.816880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.816882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.817101 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:41.817286 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:41.817574 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 04:56:41.817719 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:56:41.818522 7626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:41.900657 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 7436
I20260809 04:56:41.927450 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 7641
Aug 09 04:56:41 dist-test-slave-21b9 krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:56:41.942739 8086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:41.928446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54392 (local address 127.29.243.194:35659)
0809 04:56:41.928985 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:41.928993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:41.929019 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:56:41.929098 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:41.929105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:41.929323 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:41.929449 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:41.930513 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.931701 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.932957 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:41.933561 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:41.933672 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:41.937463 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:41.937475 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:41.937491 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:41.937536 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:41.940344 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.940964 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.940971 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.940975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.941098 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:41.941375 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:41.941378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:41.941380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:41.941721 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:41.941968 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:41.942179 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 04:56:41.942416 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":247,"threads_started":1}
I20260809 04:56:41.944118 7826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:42.028950 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 7641
I20260809 04:56:42.052589 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 7839
Aug 09 04:56:42 dist-test-slave-21b9 krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:56:42.069485 8090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:42.053780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.29.243.195:32823)
0809 04:56:42.054514 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:42.054521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:42.054555 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:56:42.054622 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:42.054629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:42.054862 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:42.055003 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:42.056127 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:42.057209 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:42.058474 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:42.059091 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:42.059201 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:42.063497 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:42.063511 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:42.063528 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:42.063569 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:42.066598 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:42.067612 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:42.067621 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:42.067623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:42.067753 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:42.068099 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:42.068102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:42.068104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:42.068461 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:42.068700 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:42.068893 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 04:56:42.069164 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":352,"threads_started":1}
I20260809 04:56:42.070757 8038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:42.151307 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 7839
I20260809 04:56:42.176915 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 7281
I20260809 04:56:42.178148 7384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:42.281108 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 7281
2026-08-09T04:56:42Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4156 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:56:42 dist-test-slave-21b9 krb5kdc[8097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:56:42 dist-test-slave-21b9 krb5kdc[8097](info): set up 2 sockets
Aug 09 04:56:42 dist-test-slave-21b9 krb5kdc[8097](info): commencing operation
krb5kdc: starting...
W20260809 04:56:43.499811 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:56:43 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:56:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:43Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.254.keytab.
Entry for principal oryx/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.254.keytab.
I20260809 04:56:43.744349 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:34625
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.254.keytab
--principal=oryx/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:34625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:44.078037 8114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:44.078359 8114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:44.078483 8114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:44.085789 8114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:44.085894 8114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:44.085951 8114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:44.086043 8114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:44.086100 8114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:44.096249 8114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:34625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.254.keytab
--principal=oryx/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:34625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:44.097628 8114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:44.098948 8114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:44.110445 8122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:44.110540 8125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:44.110496 8123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:44.111543 8114 server_base.cc:1034] running on GCE node
I20260809 04:56:44.112308 8114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:44.113751 8114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:44.114964 8114 hybrid_clock.cc:648] HybridClock initialized: now 1786251404114902 us; error 52 us; skew 500 ppm
I20260809 04:56:44.115324 8114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:44 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251404, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:44.118453 8114 init.cc:377] Logged in from keytab as oryx/127.29.243.254@KRBTEST.COM (short username oryx)
I20260809 04:56:44.120070 8114 webserver.cc:460] Webserver started at http://127.29.243.254:40521/ using document root <none> and password file <none>
I20260809 04:56:44.120774 8114 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:44.120894 8114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:44.121199 8114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:44.124248 8114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "1dbf05d2d18743ec936aebcbe9ab5c80"
format_stamp: "Formatted at 2026-08-09 04:56:44 on dist-test-slave-21b9"
server_key: "698da97a29fb257c3d576898c6e2645a"
server_key_iv: "c8713a0c590aa56bd8c8a28fcf3f725d"
server_key_version: "encryptionkey@0"
I20260809 04:56:44.125016 8114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "1dbf05d2d18743ec936aebcbe9ab5c80"
format_stamp: "Formatted at 2026-08-09 04:56:44 on dist-test-slave-21b9"
server_key: "698da97a29fb257c3d576898c6e2645a"
server_key_iv: "c8713a0c590aa56bd8c8a28fcf3f725d"
server_key_version: "encryptionkey@0"
I20260809 04:56:44.130262 8114 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:56:44.133751 8136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:44.135140 8114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:44.135353 8114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "1dbf05d2d18743ec936aebcbe9ab5c80"
format_stamp: "Formatted at 2026-08-09 04:56:44 on dist-test-slave-21b9"
server_key: "698da97a29fb257c3d576898c6e2645a"
server_key_iv: "c8713a0c590aa56bd8c8a28fcf3f725d"
server_key_version: "encryptionkey@0"
I20260809 04:56:44.135535 8114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:44.149493 8114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:44.153581 8114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:44.153836 8114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:44.171890 8114 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:34625
I20260809 04:56:44.171927 8209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:34625 every 8 connection(s)
I20260809 04:56:44.173300 8114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:44.177217 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 8114
I20260809 04:56:44.177263 8211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:44.177471 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:44.178278 30671 external_mini_cluster.cc:1428] Setting key 43a7835003d10f56177d42b2ecc84e70
I20260809 04:56:44.191164 8211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80: Bootstrap starting.
Aug 09 04:56:44 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:44.195013 8211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:44.196411 8211 log.cc:824] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:44.199673 8211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80: No bootstrap required, opened a new log
I20260809 04:56:44.202417 8220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:44.180343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.29.243.254:34625)
0809 04:56:44.181130 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:44.181141 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:44.181182 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:56:44.183445 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:44.183471 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:44.183858 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:44.184181 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:44.185971 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:44.187293 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:44.189070 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:44.189725 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:44.189852 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:44.194363 (+ 4511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:44.194374 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:44.194399 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:44.194457 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:44.198086 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:44.198716 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:44.198725 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:44.198734 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:44.198942 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:44.199294 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:44.199300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:44.199303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:44.200021 (+ 718us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:44.200369 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:44.200657 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 04:56:44.201080 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":352,"threads_started":1}
I20260809 04:56:44.208066 8211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } }
I20260809 04:56:44.208393 8211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:44.208472 8211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dbf05d2d18743ec936aebcbe9ab5c80, State: Initialized, Role: FOLLOWER
I20260809 04:56:44.209041 8211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } }
I20260809 04:56:44.209301 8211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:44.209450 8211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:44.209626 8211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:44.211791 8211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } }
I20260809 04:56:44.212277 8211 leader_election.cc:302] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dbf05d2d18743ec936aebcbe9ab5c80; no voters:
I20260809 04:56:44.212814 8211 leader_election.cc:288] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:44.212960 8224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:44.213655 8224 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 1 LEADER]: Becoming Leader. State: Replica: 1dbf05d2d18743ec936aebcbe9ab5c80, State: Running, Role: LEADER
I20260809 04:56:44.214184 8224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } }
I20260809 04:56:44.214887 8211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:44.217374 8226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dbf05d2d18743ec936aebcbe9ab5c80. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } }
I20260809 04:56:44.217370 8225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } }
I20260809 04:56:44.217768 8225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:44.217768 8226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:44.224645 8235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:44.229056 8235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:44.237723 8235 catalog_manager.cc:1347] Generated new cluster ID: c12f80fb769e40f385feb51c1cf0ac40
I20260809 04:56:44.237841 8235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:44.257572 8235 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:44.259519 8235 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:44.272360 8235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80: Generated new TSK 0
I20260809 04:56:44.273468 8235 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab.
Entry for principal oryx/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab.
I20260809 04:56:44.345139 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:34625
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab
--principal=oryx/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:44.687676 8256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:44.687980 8256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:44.688122 8256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:44.695243 8256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:44.695384 8256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:44.695544 8256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:56:44.705369 8256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab
--principal=oryx/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:34625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:44.706868 8256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:44.708242 8256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:44.721885 8265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:44.721792 8266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:44.721825 8268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:44.722736 8256 server_base.cc:1034] running on GCE node
I20260809 04:56:44.723364 8256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:44.724267 8256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:44.725476 8256 hybrid_clock.cc:648] HybridClock initialized: now 1786251404725415 us; error 52 us; skew 500 ppm
I20260809 04:56:44.725769 8256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:44 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251404, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:44.728881 8256 init.cc:377] Logged in from keytab as oryx/127.29.243.193@KRBTEST.COM (short username oryx)
I20260809 04:56:44.730257 8256 webserver.cc:460] Webserver started at http://127.29.243.193:37273/ using document root <none> and password file <none>
I20260809 04:56:44.730911 8256 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:44.731040 8256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:44.731376 8256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:44.734222 8256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "97b24491f3874d1fa98090dc7e4ee226"
format_stamp: "Formatted at 2026-08-09 04:56:44 on dist-test-slave-21b9"
server_key: "08b4d7661f26c0dfdc5639cc7586be7c"
server_key_iv: "4254f25476a05074135cefb4882f3c5a"
server_key_version: "encryptionkey@0"
I20260809 04:56:44.735072 8256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "97b24491f3874d1fa98090dc7e4ee226"
format_stamp: "Formatted at 2026-08-09 04:56:44 on dist-test-slave-21b9"
server_key: "08b4d7661f26c0dfdc5639cc7586be7c"
server_key_iv: "4254f25476a05074135cefb4882f3c5a"
server_key_version: "encryptionkey@0"
I20260809 04:56:44.740320 8256 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 04:56:44.744062 8276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:44.745679 8256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:56:44.745846 8256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "97b24491f3874d1fa98090dc7e4ee226"
format_stamp: "Formatted at 2026-08-09 04:56:44 on dist-test-slave-21b9"
server_key: "08b4d7661f26c0dfdc5639cc7586be7c"
server_key_iv: "4254f25476a05074135cefb4882f3c5a"
server_key_version: "encryptionkey@0"
I20260809 04:56:44.746034 8256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:44.786383 8256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:44.790675 8256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:44.790962 8256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:44.791972 8256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:44.793358 8256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:44.793438 8256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:44.793558 8256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:44.793612 8256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:44.824715 8256 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:45623
I20260809 04:56:44.824757 8474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:45623 every 8 connection(s)
I20260809 04:56:44.825953 8256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:56:44.830843 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 8256
I20260809 04:56:44.831054 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:56:44.831758 30671 external_mini_cluster.cc:1428] Setting key 229efd4c350ceaf5f67c13e65fac9456
Aug 09 04:56:44 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251404, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:44.846542 8477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:44.828567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:38455 (local address 127.29.243.254:34625)
0809 04:56:44.829027 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:44.829032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:44.830038 (+ 1006us) server_negotiation.cc:407] Connection header received
0809 04:56:44.831957 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:44.831963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:44.832053 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:44.832187 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:44.833843 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:44.834887 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:44.836458 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:44.836985 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:44.837120 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:44.840553 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:44.840559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:44.840563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:44.840600 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:44.843403 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:44.844293 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:44.844297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:44.844300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:44.844427 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:44.844886 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:44.844889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:44.844891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:44.845148 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:44.845256 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:44.846093 (+ 837us) server_negotiation.cc:299] Negotiation successful
0809 04:56:44.846262 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":223,"threads_started":1}
I20260809 04:56:44.847887 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:44.829138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.193:38455)
0809 04:56:44.829869 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:44.829929 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:44.831699 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:44.832413 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:44.832421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:44.833046 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:44.833689 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:44.833710 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:44.835019 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:44.835023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:44.836298 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:44.836315 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:44.837263 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:44.837268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:44.837482 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:44.838137 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:44.838160 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:44.840335 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:44.843545 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:44.843553 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:44.843568 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:44.844171 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:44.844525 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:44.844530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:44.844533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:44.844769 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:44.845271 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:44.845279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:44.845663 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 04:56:44.845949 (+ 286us) client_negotiation.cc:240] Negotiation successful
0809 04:56:44.846469 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":296,"threads_started":1}
I20260809 04:56:44.849454 8475 heartbeater.cc:347] Connected to a master server at 127.29.243.254:34625
I20260809 04:56:44.849805 8475 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:44.850754 8475 heartbeater.cc:511] Master 127.29.243.254:34625 requested a full tablet report, sending...
I20260809 04:56:44.852948 8162 ts_manager.cc:194] Registered new tserver with Master: 97b24491f3874d1fa98090dc7e4ee226 (127.29.243.193:45623)
I20260809 04:56:44.854429 8162 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:38455
WARNING: no policy specified for oryx/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab.
Entry for principal oryx/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab.
I20260809 04:56:44.886471 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:34625
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab
--principal=oryx/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:45.222608 8482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:45.222892 8482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:45.222987 8482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:45.230319 8482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:45.230430 8482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:45.230542 8482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:56:45.240252 8482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab
--principal=oryx/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:34625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:45.241508 8482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:45.242810 8482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:45.255781 8488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:45.255857 8489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:45.255815 8494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:45.257041 8482 server_base.cc:1034] running on GCE node
I20260809 04:56:45.257664 8482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:45.258607 8482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:45.259832 8482 hybrid_clock.cc:648] HybridClock initialized: now 1786251405259782 us; error 38 us; skew 500 ppm
I20260809 04:56:45.260179 8482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:45 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251405, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:45.263262 8482 init.cc:377] Logged in from keytab as oryx/127.29.243.194@KRBTEST.COM (short username oryx)
I20260809 04:56:45.264775 8482 webserver.cc:460] Webserver started at http://127.29.243.194:37773/ using document root <none> and password file <none>
I20260809 04:56:45.265487 8482 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:45.265609 8482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:45.265908 8482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:45.268985 8482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "7ff56221a0be4ba392f866e0ea37cfa6"
format_stamp: "Formatted at 2026-08-09 04:56:45 on dist-test-slave-21b9"
server_key: "2e1deab75d59d657c88ec4ca17fb2ebd"
server_key_iv: "9ff870df1adb7bad63a3983916f45201"
server_key_version: "encryptionkey@0"
I20260809 04:56:45.269770 8482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "7ff56221a0be4ba392f866e0ea37cfa6"
format_stamp: "Formatted at 2026-08-09 04:56:45 on dist-test-slave-21b9"
server_key: "2e1deab75d59d657c88ec4ca17fb2ebd"
server_key_iv: "9ff870df1adb7bad63a3983916f45201"
server_key_version: "encryptionkey@0"
I20260809 04:56:45.274901 8482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:56:45.278456 8509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:45.279807 8482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:56:45.279991 8482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "7ff56221a0be4ba392f866e0ea37cfa6"
format_stamp: "Formatted at 2026-08-09 04:56:45 on dist-test-slave-21b9"
server_key: "2e1deab75d59d657c88ec4ca17fb2ebd"
server_key_iv: "9ff870df1adb7bad63a3983916f45201"
server_key_version: "encryptionkey@0"
I20260809 04:56:45.280179 8482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:45.314253 8482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:45.318830 8482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:45.319109 8482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:45.320190 8482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:45.321660 8482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:45.321739 8482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:45.321866 8482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:45.321918 8482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:45.352811 8482 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:41757
I20260809 04:56:45.352860 8680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:41757 every 8 connection(s)
I20260809 04:56:45.354080 8482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:56:45.359912 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 8482
I20260809 04:56:45.360107 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:56:45.360685 30671 external_mini_cluster.cc:1428] Setting key 0437c09d7773fc7de2a4eee03dd10497
Aug 09 04:56:45 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251405, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:45.375541 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:45.356673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:35739 (local address 127.29.243.254:34625)
0809 04:56:45.357109 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:45.357114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:45.358130 (+ 1016us) server_negotiation.cc:407] Connection header received
0809 04:56:45.360039 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:45.360045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:45.360128 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:45.360231 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:45.362022 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:45.362969 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:45.365223 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:45.365684 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:45.365790 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:45.369813 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:45.369819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:45.369822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:45.369859 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:45.372371 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:45.373196 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:45.373203 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:45.373206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:45.373406 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:45.373779 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:45.373783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:45.373786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:45.374121 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:45.374267 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:45.374911 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 04:56:45.375152 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":208,"threads_started":1}
I20260809 04:56:45.376268 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:45.357275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.194:35739)
0809 04:56:45.357954 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:45.358002 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:45.359751 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:45.360511 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:45.360523 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:45.361259 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:45.361857 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:45.361875 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:45.363136 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:45.363141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:45.365064 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:45.365079 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:45.365915 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:45.365921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:45.366207 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:45.366900 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:45.366934 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:45.369560 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:45.372499 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:45.372508 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:45.372524 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:45.373050 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:45.373502 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:45.373505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:45.373507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:45.373678 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:45.374221 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:45.374226 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:45.374540 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 04:56:45.374760 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 04:56:45.375151 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":259,"threads_started":1}
I20260809 04:56:45.377576 8681 heartbeater.cc:347] Connected to a master server at 127.29.243.254:34625
I20260809 04:56:45.377866 8681 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:45.378590 8681 heartbeater.cc:511] Master 127.29.243.254:34625 requested a full tablet report, sending...
I20260809 04:56:45.380033 8162 ts_manager.cc:194] Registered new tserver with Master: 7ff56221a0be4ba392f866e0ea37cfa6 (127.29.243.194:41757)
I20260809 04:56:45.380857 8162 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:35739
WARNING: no policy specified for oryx/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab.
Entry for principal oryx/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab.
I20260809 04:56:45.418612 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:34625
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab
--principal=oryx/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:45.756819 8687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:45.757114 8687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:45.757249 8687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:45.764410 8687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:45.764530 8687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:45.764693 8687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:56:45.774850 8687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab
--principal=oryx/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:34625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:45.776260 8687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:45.777606 8687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:45.790692 8698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:45.790755 8699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:45.790728 8706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:45.791473 8687 server_base.cc:1034] running on GCE node
I20260809 04:56:45.792054 8687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:45.792948 8687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:45.794184 8687 hybrid_clock.cc:648] HybridClock initialized: now 1786251405794123 us; error 45 us; skew 500 ppm
I20260809 04:56:45.794543 8687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:45 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251405, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:45.797801 8687 init.cc:377] Logged in from keytab as oryx/127.29.243.195@KRBTEST.COM (short username oryx)
I20260809 04:56:45.799220 8687 webserver.cc:460] Webserver started at http://127.29.243.195:39131/ using document root <none> and password file <none>
I20260809 04:56:45.799925 8687 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:45.800045 8687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:45.800379 8687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:45.803246 8687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "0490a08d31ba4a588b0a4129ab632520"
format_stamp: "Formatted at 2026-08-09 04:56:45 on dist-test-slave-21b9"
server_key: "45fda7d624ec6172b9d99e4bf3c9044e"
server_key_iv: "8b22a27e6cc51a8456045876859d966a"
server_key_version: "encryptionkey@0"
I20260809 04:56:45.804065 8687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "0490a08d31ba4a588b0a4129ab632520"
format_stamp: "Formatted at 2026-08-09 04:56:45 on dist-test-slave-21b9"
server_key: "45fda7d624ec6172b9d99e4bf3c9044e"
server_key_iv: "8b22a27e6cc51a8456045876859d966a"
server_key_version: "encryptionkey@0"
I20260809 04:56:45.809098 8687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 04:56:45.812609 8717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:45.814137 8687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:56:45.814309 8687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "0490a08d31ba4a588b0a4129ab632520"
format_stamp: "Formatted at 2026-08-09 04:56:45 on dist-test-slave-21b9"
server_key: "45fda7d624ec6172b9d99e4bf3c9044e"
server_key_iv: "8b22a27e6cc51a8456045876859d966a"
server_key_version: "encryptionkey@0"
I20260809 04:56:45.814529 8687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:45.828825 8687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:45.833088 8687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:45.833328 8687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:45.834169 8687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:45.835549 8687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:45.835623 8687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:45.835716 8687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:45.835760 8687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:45.858008 8475 heartbeater.cc:503] Master 127.29.243.254:34625 was elected leader, sending a full tablet report...
I20260809 04:56:45.866513 8687 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:37453
I20260809 04:56:45.866554 8890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:37453 every 8 connection(s)
I20260809 04:56:45.867818 8687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:56:45.871791 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 8687
I20260809 04:56:45.872010 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:56:45.872587 30671 external_mini_cluster.cc:1428] Setting key 6fd78dfc0ec64b5893f3b461d9e32e64
Aug 09 04:56:45 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251405, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:45.887769 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:45.870451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:47611 (local address 127.29.243.254:34625)
0809 04:56:45.870624 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:45.870628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:45.871791 (+ 1163us) server_negotiation.cc:407] Connection header received
0809 04:56:45.874040 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:45.874045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:45.874131 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:45.874217 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:45.876138 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:45.876865 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:45.878386 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:45.878770 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:45.878880 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:45.882136 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:45.882140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:45.882144 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:45.882177 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:45.884698 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:45.885430 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:45.885435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:45.885437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:45.885556 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:45.885982 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:45.885986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:45.885988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:45.886257 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:45.886412 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:45.887161 (+ 749us) server_negotiation.cc:299] Negotiation successful
0809 04:56:45.887440 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:56:45.888715 8893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:45.870813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.195:47611)
0809 04:56:45.871605 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:45.871657 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:45.873734 (+ 2077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:45.874467 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:45.874476 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:45.875184 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:45.875961 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:45.875988 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:45.877015 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:45.877019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:45.878233 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:45.878245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:45.878992 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:45.878999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:45.879211 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:45.879867 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:45.879890 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:45.881942 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:45.884843 (+ 2901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:45.884852 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:45.884867 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:45.885306 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:45.885657 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:45.885662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:45.885666 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:45.885868 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:45.886385 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:45.886393 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:45.886733 (+ 340us) client_negotiation.cc:769] Sending connection context
0809 04:56:45.887018 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 04:56:45.887541 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":279,"threads_started":1}
I20260809 04:56:45.890074 8891 heartbeater.cc:347] Connected to a master server at 127.29.243.254:34625
I20260809 04:56:45.890419 8891 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:45.891220 8891 heartbeater.cc:511] Master 127.29.243.254:34625 requested a full tablet report, sending...
I20260809 04:56:45.892726 8162 ts_manager.cc:194] Registered new tserver with Master: 0490a08d31ba4a588b0a4129ab632520 (127.29.243.195:37453)
I20260809 04:56:45.893513 8162 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:47611
I20260809 04:56:45.900178 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.243.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.253.keytab.
Entry for principal oryx/127.29.243.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.253.keytab.
I20260809 04:56:45.955538 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.253:45079
--webserver_interface=127.29.243.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.253.keytab
--principal=oryx/127.29.243.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:34625,127.29.243.253:45079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:46.293521 8897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:46.293794 8897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:46.293919 8897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:46.301080 8897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:46.301186 8897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:46.301244 8897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:46.301318 8897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:46.301383 8897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:46.311468 8897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:34625,127.29.243.253:45079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.253.keytab
--principal=oryx/127.29.243.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.253:45079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.243.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
W20260809 04:56:46.311805 8897 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.243.254:34625,127.29.243.253:45079'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 04:56:46.312940 8897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:46.314195 8897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:46.325567 8903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:46.325536 8904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:46.325496 8907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:46.326579 8897 server_base.cc:1034] running on GCE node
I20260809 04:56:46.327318 8897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:46.328707 8897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:46.329932 8897 hybrid_clock.cc:648] HybridClock initialized: now 1786251406329855 us; error 51 us; skew 500 ppm
I20260809 04:56:46.330262 8897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:46 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251406, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:46.333416 8897 init.cc:377] Logged in from keytab as oryx/127.29.243.253@KRBTEST.COM (short username oryx)
I20260809 04:56:46.334864 8897 webserver.cc:460] Webserver started at http://127.29.243.253:38685/ using document root <none> and password file <none>
I20260809 04:56:46.335597 8897 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:46.335701 8897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:46.336004 8897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:46.338855 8897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data/instance:
uuid: "5be2fa38862348589240908d3ba94459"
format_stamp: "Formatted at 2026-08-09 04:56:46 on dist-test-slave-21b9"
server_key: "9d987cbc98d66bb71e44465e53627efe"
server_key_iv: "47938606509ae22e01ec247f25387e37"
server_key_version: "encryptionkey@0"
I20260809 04:56:46.339695 8897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/wal/instance:
uuid: "5be2fa38862348589240908d3ba94459"
format_stamp: "Formatted at 2026-08-09 04:56:46 on dist-test-slave-21b9"
server_key: "9d987cbc98d66bb71e44465e53627efe"
server_key_iv: "47938606509ae22e01ec247f25387e37"
server_key_version: "encryptionkey@0"
I20260809 04:56:46.344753 8897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:56:46.348104 8921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:46.349438 8897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:56:46.349591 8897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/wal
uuid: "5be2fa38862348589240908d3ba94459"
format_stamp: "Formatted at 2026-08-09 04:56:46 on dist-test-slave-21b9"
server_key: "9d987cbc98d66bb71e44465e53627efe"
server_key_iv: "47938606509ae22e01ec247f25387e37"
server_key_version: "encryptionkey@0"
I20260809 04:56:46.349774 8897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:46.363337 8897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:46.367447 8897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:46.367692 8897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:46.383378 8681 heartbeater.cc:503] Master 127.29.243.254:34625 was elected leader, sending a full tablet report...
I20260809 04:56:46.386049 8897 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.253:45079
I20260809 04:56:46.386081 9002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.253:45079 every 8 connection(s)
I20260809 04:56:46.387522 8897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
I20260809 04:56:46.389201 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 8897
I20260809 04:56:46.392035 9003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:46.400066 9003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:56:46 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.243.253@KRBTEST.COM
Aug 09 04:56:46 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251406, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.253@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:46.412107 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.391525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.29.243.253:45079)
0809 04:56:46.392255 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:46.392266 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:46.392310 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:56:46.393857 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:46.393878 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:46.394245 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:46.394572 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:46.396411 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.397809 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.399539 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.400232 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.400363 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.405183 (+ 4820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:46.405190 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:46.405202 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:46.405236 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:46.408056 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.408775 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.408782 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.408788 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.408969 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.409364 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.409370 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.409372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.409918 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:46.410209 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:46.410666 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 04:56:46.411056 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":322,"threads_started":1}
I20260809 04:56:46.417395 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.403026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.29.243.254:34625)
0809 04:56:46.403583 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:46.403590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:46.404105 (+ 515us) server_negotiation.cc:407] Connection header received
0809 04:56:46.404293 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:46.404297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:46.404362 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:46.404456 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:46.405716 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.406447 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.407894 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.408294 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.408623 (+ 329us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.411909 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:46.411914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:46.411917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:46.411961 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:46.414695 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.415400 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.415405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.415409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.415530 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.415887 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.415892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.415896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.416207 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:46.416359 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:46.416813 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 04:56:46.417032 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":294,"threads_started":1}
I20260809 04:56:46.417394 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.403044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:43416)
0809 04:56:46.403953 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:46.403996 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:46.404164 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:46.404544 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:46.404551 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:46.405257 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:46.405563 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.405573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.406582 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.406587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.407747 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.407759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.408651 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.408658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.408825 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.409389 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:46.409407 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:46.411726 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:46.414838 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.414849 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.414856 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.415248 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.415636 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.415641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.415645 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.415774 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.416336 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:46.416344 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:46.416521 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 04:56:46.416681 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 04:56:46.416986 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":518,"threads_started":1}
I20260809 04:56:46.419134 9003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:56:46 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251406, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.253@KRBTEST.COM for oryx/127.29.243.253@KRBTEST.COM
I20260809 04:56:46.430406 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.419904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:45079 (local address 127.0.0.1:36884)
0809 04:56:46.420075 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:46.420094 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:46.420185 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:46.420469 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:46.420473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:46.420789 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:46.420911 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.420919 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.421850 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.421854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.422687 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.422698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.423360 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.423366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.423517 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.423958 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:46.423973 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:46.425738 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:46.428282 (+ 2544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.428287 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.428291 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.428620 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.428958 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.428963 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.428966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.429103 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.429570 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:46.429574 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:46.429699 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 04:56:46.429809 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 04:56:46.430069 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 04:56:46.430593 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.420107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.29.243.253:45079)
0809 04:56:46.420251 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:46.420256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:46.420271 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:56:46.420308 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:46.420313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:46.420365 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:46.420440 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:46.421037 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.421715 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.422818 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.423201 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.423326 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.425894 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:46.425899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:46.425902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:46.425929 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:46.428166 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.428724 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.428729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.428732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.428857 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.429200 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.429205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.429209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.429455 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:46.429586 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:46.429904 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 04:56:46.430230 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":46}
I20260809 04:56:46.435081 9003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: Bootstrap starting.
I20260809 04:56:46.438297 9003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:46.439383 9003 log.cc:824] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:46.442121 9003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: No bootstrap required, opened a new log
I20260809 04:56:46.449112 9003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } }
I20260809 04:56:46.449440 9003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:46.449539 9003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5be2fa38862348589240908d3ba94459, State: Initialized, Role: FOLLOWER
I20260809 04:56:46.450374 9003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } }
I20260809 04:56:46.450695 9016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:46.451016 9016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } }
I20260809 04:56:46.451834 9017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } } }
I20260809 04:56:46.452255 9017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:56:46.452354 9003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:46.452884 9016 leader_election.cc:288] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dbf05d2d18743ec936aebcbe9ab5c80 (127.29.243.254:34625)
I20260809 04:56:46.453575 8176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5be2fa38862348589240908d3ba94459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" is_pre_election: true
I20260809 04:56:46.454815 8927 leader_election.cc:302] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 5be2fa38862348589240908d3ba94459; no voters: 1dbf05d2d18743ec936aebcbe9ab5c80
I20260809 04:56:46.455296 9016 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 04:56:46.462131 9038 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:56:46.462265 9038 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:56:46.464856 8897 master_runner.cc:428] Detected that this master 5be2fa38862348589240908d3ba94459 is joining an existing cluster
I20260809 04:56:46.464938 8897 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.243.253:45079
I20260809 04:56:46.496240 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.470218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.29.243.254:34625)
0809 04:56:46.470403 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:46.470409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:46.480728 (+ 10319us) server_negotiation.cc:407] Connection header received
0809 04:56:46.480913 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:46.480918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:46.481018 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:46.481123 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:46.482749 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.483782 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.485859 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.486465 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.486576 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.489264 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:46.489271 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:46.489275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:46.489317 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:46.493003 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.493789 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.493794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.493798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.493961 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.494445 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.494450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.494454 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.494824 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:46.494946 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:46.495613 (+ 667us) server_negotiation.cc:299] Negotiation successful
0809 04:56:46.495872 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:56:46.497323 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.470228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.29.243.253:45079)
0809 04:56:46.470410 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:46.470416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:46.482899 (+ 12483us) server_negotiation.cc:407] Connection header received
0809 04:56:46.483331 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:46.483337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:46.483407 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:46.483507 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:46.484933 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.486105 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.487844 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.488541 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.488630 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.491037 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:46.491043 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:46.491048 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:46.491080 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:46.494451 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.495173 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.495178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.495182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.495351 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.495776 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.495781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.495784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.496153 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:46.496305 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:46.496779 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 04:56:46.496979 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:56:46.497822 9050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.469723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:43432)
0809 04:56:46.480577 (+ 10854us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:46.480601 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:46.480785 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:46.481333 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:46.481338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:46.482104 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:46.482534 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.482547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.483954 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.483959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.485687 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.485701 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.486884 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.486890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.487096 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.487807 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:46.487830 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:46.489024 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:46.493164 (+ 4140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.493169 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.493177 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.493645 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.494131 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.494136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.494139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.494310 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.495056 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:46.495061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:46.495293 (+ 232us) client_negotiation.cc:769] Sending connection context
0809 04:56:46.496832 (+ 1539us) client_negotiation.cc:240] Negotiation successful
0809 04:56:46.497143 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10659,"mutex_wait_us":26,"thread_start_us":13187,"threads_started":1}
I20260809 04:56:46.498263 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.469741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:45079 (local address 127.0.0.1:36894)
0809 04:56:46.482804 (+ 13063us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:46.482822 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:46.483164 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:46.483644 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:46.483652 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:46.484451 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:46.484779 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.484791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.486275 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.486280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.487700 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.487716 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.488950 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.488957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.489175 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.489770 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:46.489791 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:46.490832 (+ 1041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:46.494590 (+ 3758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.494595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.494598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.495071 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.495505 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.495509 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.495512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.495664 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.496392 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:46.496398 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:46.496543 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 04:56:46.497582 (+ 1039us) client_negotiation.cc:240] Negotiation successful
0809 04:56:46.497856 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12930,"mutex_wait_us":1,"thread_start_us":10493,"threads_started":1}
WARNING: no policy specified for oryx/127.29.243.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.252@KRBTEST.COM" created.
Aug 09 04:56:46 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251404, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.254@KRBTEST.COM for oryx/127.29.243.253@KRBTEST.COM
I20260809 04:56:46.526132 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.504492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36896 (local address 127.29.243.253:45079)
0809 04:56:46.504837 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:46.504843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:46.510895 (+ 6052us) server_negotiation.cc:407] Connection header received
0809 04:56:46.511265 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:46.511298 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:46.511381 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:46.511467 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:46.512598 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.513324 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.514590 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.515069 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.515177 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.519087 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:46.519093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:46.519098 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:46.519137 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:46.523025 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.523900 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.523905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.523908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.524097 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.524532 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.524537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.524539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.524847 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:46.524987 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:46.525569 (+ 582us) server_negotiation.cc:299] Negotiation successful
0809 04:56:46.525806 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220}
I20260809 04:56:46.526229 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.504398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:45079 (local address 127.0.0.1:36896)
0809 04:56:46.510742 (+ 6344us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:46.510795 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:46.511050 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:46.511538 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:46.511546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:46.512213 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:46.512443 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.512453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.513452 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.513458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.514445 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.514458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.515232 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.515238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.515496 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.516150 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:46.516174 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:46.518863 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:46.523172 (+ 4309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.523185 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.523193 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.523777 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.524219 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.524223 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.524227 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.524397 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.524988 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:46.524997 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:46.525206 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 04:56:46.525426 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 04:56:46.525762 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6100,"thread_start_us":5936,"threads_started":1}
I20260809 04:56:46.528693 8162 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.243.253:45079: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5be2fa38862348589240908d3ba94459"
member_type: NON_VOTER
last_known_addr {
host: "127.29.243.253"
port: 45079
}
attrs {
promote: true
}
}
dest_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80"
cas_config_opid_index: -1
I20260809 04:56:46.530418 8162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } }
I20260809 04:56:46.535241 9058 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5be2fa38862348589240908d3ba94459 (127.29.243.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } } }
I20260809 04:56:46.536765 8965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:46.540382 9058 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.243.253:45079
I20260809 04:56:46.540441 8965 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 FOLLOWER]: Refusing update from remote peer 1dbf05d2d18743ec936aebcbe9ab5c80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.252.keytab.
Entry for principal oryx/127.29.243.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.252.keytab.
I20260809 04:56:46.541661 8897 master.cc:537] Master@127.29.243.253:45079 shutting down...
I20260809 04:56:46.543318 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.252:39747
--webserver_interface=127.29.243.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.252.keytab
--principal=oryx/127.29.243.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:34625,127.29.243.253:45079,127.29.243.252:39747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:56:46.547883 9057 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:46.548782 9060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } } }
I20260809 04:56:46.549185 9060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: This master's current role is: LEADER
W20260809 04:56:46.551584 8141 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 04:56:46.557019 8897 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:46.557438 8897 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:46.557626 8897 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: stopping tablet replica
W20260809 04:56:46.559000 8141 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 -> Peer 5be2fa38862348589240908d3ba94459 (127.29.243.253:45079): Couldn't send request to peer 5be2fa38862348589240908d3ba94459. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 04:56:46.564924 8897 master.cc:552] Master@127.29.243.253:45079 shutdown complete.
I20260809 04:56:46.565075 8897 master_runner.cc:305] Clearing existing system tablet
I20260809 04:56:46.565546 8897 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:46.565644 8897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:46.568751 8897 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:56:46.570534 9068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:46.570911 8897 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:46.571370 8897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:56:46.571514 8897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/wal
uuid: "5be2fa38862348589240908d3ba94459"
format_stamp: "Formatted at 2026-08-09 04:56:46 on dist-test-slave-21b9"
server_key: "9d987cbc98d66bb71e44465e53627efe"
server_key_iv: "47938606509ae22e01ec247f25387e37"
server_key_version: "encryptionkey@0"
I20260809 04:56:46.577195 8897 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:46.584631 8897 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 04:56:46.584888 8897 log.cc:1196] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 04:56:46.585382 8897 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: Deleting consensus metadata
I20260809 04:56:46.585696 8897 master_runner.cc:315] Copying system tablet from 127.29.243.254:34625
I20260809 04:56:46.589129 8897 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: tablet copy: Beginning tablet copy session from remote peer at address 127.29.243.254:34625
I20260809 04:56:46.608618 9077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.590185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52140)
0809 04:56:46.590714 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:46.590738 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:46.590968 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:46.591607 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:46.591612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:46.592191 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:46.592466 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.592480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.593992 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.593998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.595522 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:46.595538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.596742 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.596748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:46.596946 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.600269 (+ 3323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:46.600293 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:46.601365 (+ 1072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:46.605613 (+ 4248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.605618 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.605623 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.606126 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.606604 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:46.606609 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:46.606613 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:46.606795 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:46.607494 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:46.607500 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:46.607682 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 04:56:46.607858 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 04:56:46.608181 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":505,"threads_started":1}
I20260809 04:56:46.608659 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:46.590510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.29.243.254:34625)
0809 04:56:46.590697 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:46.590703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:46.590843 (+ 140us) server_negotiation.cc:407] Connection header received
0809 04:56:46.591128 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:46.591134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:46.591236 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:46.591386 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:46.592646 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.593781 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.595693 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:46.596375 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:46.596474 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:46.601624 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:46.601632 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:46.601637 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:46.601689 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:46.605462 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.606278 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.606284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.606288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.606454 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:46.606941 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:46.606947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:46.606950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:46.607359 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:46.607520 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:46.607981 (+ 461us) server_negotiation.cc:299] Negotiation successful
0809 04:56:46.608306 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:56:46.610023 8189 tablet_copy_service.cc:140] P 1dbf05d2d18743ec936aebcbe9ab5c80: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5be2fa38862348589240908d3ba94459 ({username='oryx', principal='oryx/127.29.243.253@KRBTEST.COM'} at 127.0.0.1:52140)
I20260809 04:56:46.610390 8189 tablet_copy_service.cc:161] P 1dbf05d2d18743ec936aebcbe9ab5c80: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5be2fa38862348589240908d3ba94459 at {username='oryx', principal='oryx/127.29.243.253@KRBTEST.COM'} at 127.0.0.1:52140: session id = 5be2fa38862348589240908d3ba94459-00000000000000000000000000000000
I20260809 04:56:46.615222 8189 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 04:56:46.618127 8897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 04:56:46.625119 8897 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: tablet copy: Starting download of 0 data blocks...
I20260809 04:56:46.625511 8897 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: tablet copy: Starting download of 1 WAL segments...
I20260809 04:56:46.629310 8897 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 04:56:46.633793 8189 tablet_copy_service.cc:342] P 1dbf05d2d18743ec936aebcbe9ab5c80: Request end of tablet copy session 5be2fa38862348589240908d3ba94459-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.243.253@KRBTEST.COM'} at 127.0.0.1:52140
I20260809 04:56:46.634038 8189 tablet_copy_service.cc:433] P 1dbf05d2d18743ec936aebcbe9ab5c80: ending tablet copy session 5be2fa38862348589240908d3ba94459-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5be2fa38862348589240908d3ba94459
I20260809 04:56:46.646533 8897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:46.649192 9078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:46.654596 9079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:46.661398 9082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:46.661895 8897 server_base.cc:1034] running on GCE node
I20260809 04:56:46.662295 8897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:46.665333 8897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:46.666548 8897 hybrid_clock.cc:648] HybridClock initialized: now 1786251406666482 us; error 40 us; skew 500 ppm
I20260809 04:56:46.666774 8897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:46 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251406, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:46.668977 8897 init.cc:377] Logged in from keytab as oryx/127.29.243.253@KRBTEST.COM (short username oryx)
I20260809 04:56:46.670258 8897 webserver.cc:460] Webserver started at http://127.29.243.253:34267/ using document root <none> and password file <none>
I20260809 04:56:46.670657 8897 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:46.670747 8897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:46.673867 8897 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:56:46.676004 9090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:46.676452 8897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 04:56:46.676577 8897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/wal
uuid: "5be2fa38862348589240908d3ba94459"
format_stamp: "Formatted at 2026-08-09 04:56:46 on dist-test-slave-21b9"
server_key: "9d987cbc98d66bb71e44465e53627efe"
server_key_iv: "47938606509ae22e01ec247f25387e37"
server_key_version: "encryptionkey@0"
I20260809 04:56:46.676755 8897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:46.700516 8897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:46.704614 8897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:46.739354 8897 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.253:45079
I20260809 04:56:46.739588 9174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.253:45079 every 8 connection(s)
I20260809 04:56:46.740375 8897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
I20260809 04:56:46.748014 9175 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:56:46.752496 9175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: Bootstrap starting.
I20260809 04:56:46.778545 9175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:56:46.779155 9175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: Bootstrap complete.
I20260809 04:56:46.780424 9175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } }
I20260809 04:56:46.780725 9175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be2fa38862348589240908d3ba94459, State: Initialized, Role: LEARNER
I20260809 04:56:46.781057 9175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } }
I20260809 04:56:46.781955 9180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } } }
I20260809 04:56:46.782313 9175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:46.782439 9180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:56:46.799619 9195 catalog_manager.cc:1259] Loaded cluster ID: c12f80fb769e40f385feb51c1cf0ac40
I20260809 04:56:46.799732 9195 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: loading cluster ID for follower catalog manager: success
I20260809 04:56:46.803377 9195 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: acquiring CA information for follower catalog manager: success
I20260809 04:56:46.805912 9195 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:56:46.895999 8891 heartbeater.cc:503] Master 127.29.243.254:34625 was elected leader, sending a full tablet report...
W20260809 04:56:46.929796 9066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:46.930142 9066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:46.930365 9066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:46.940856 9066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:46.941056 9066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:46.941184 9066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:46.941344 9066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:46.941469 9066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:46.955390 9066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:34625,127.29.243.253:45079,127.29.243.252:39747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.252.keytab
--principal=oryx/127.29.243.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.252:39747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.243.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:46.957438 9066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:46.959139 9066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:46.979704 9204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:46.980126 9205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:46.980136 9208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:46.981071 9066 server_base.cc:1034] running on GCE node
I20260809 04:56:46.981856 9066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:46.983625 9066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:46.984972 9066 hybrid_clock.cc:648] HybridClock initialized: now 1786251406984801 us; error 71 us; skew 500 ppm
I20260809 04:56:46.985445 9066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:46 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251406, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:46.989979 9066 init.cc:377] Logged in from keytab as oryx/127.29.243.252@KRBTEST.COM (short username oryx)
I20260809 04:56:46.991991 9066 webserver.cc:460] Webserver started at http://127.29.243.252:46131/ using document root <none> and password file <none>
I20260809 04:56:46.992887 9066 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:46.993095 9066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:46.993505 9066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:46.997480 9066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data/instance:
uuid: "3d05306c85104851ac7150793513ba54"
format_stamp: "Formatted at 2026-08-09 04:56:46 on dist-test-slave-21b9"
server_key: "6fd8ea40a21603b55334ec79d77a1ea0"
server_key_iv: "d4ac157b6170e105ed3ba21c01cfbb6e"
server_key_version: "encryptionkey@0"
I20260809 04:56:46.998394 9066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/wal/instance:
uuid: "3d05306c85104851ac7150793513ba54"
format_stamp: "Formatted at 2026-08-09 04:56:46 on dist-test-slave-21b9"
server_key: "6fd8ea40a21603b55334ec79d77a1ea0"
server_key_iv: "d4ac157b6170e105ed3ba21c01cfbb6e"
server_key_version: "encryptionkey@0"
I20260809 04:56:47.005337 9066 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260809 04:56:47.010537 9219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:47.012076 9066 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260809 04:56:47.012310 9066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/wal
uuid: "3d05306c85104851ac7150793513ba54"
format_stamp: "Formatted at 2026-08-09 04:56:46 on dist-test-slave-21b9"
server_key: "6fd8ea40a21603b55334ec79d77a1ea0"
server_key_iv: "d4ac157b6170e105ed3ba21c01cfbb6e"
server_key_version: "encryptionkey@0"
I20260809 04:56:47.012610 9066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:47.027812 9066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:47.033603 9066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:47.034078 9066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:47.070114 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.058783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.29.243.253:45079)
0809 04:56:47.060999 (+ 2216us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.061004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.061021 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:56:47.061085 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.061089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.061161 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.061271 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:47.061980 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.064127 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.067192 (+ 3065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.068751 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.068927 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.069094 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.069547 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.069675 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2066,"thread_start_us":1873,"threads_started":1}
I20260809 04:56:47.070169 9260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.058457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:45079 (local address 127.0.0.1:60754)
0809 04:56:47.059257 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.059325 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:47.059540 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:47.061382 (+ 1842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:47.061388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:47.061424 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:47.061787 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.061801 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.064312 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.064320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.066981 (+ 2661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.067000 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.069008 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.069016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.069336 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.069407 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 04:56:47.069528 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 04:56:47.069676 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":502,"threads_started":1}
I20260809 04:56:47.072158 9131 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 04:56:47.073357 9180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dbf05d2d18743ec936aebcbe9ab5c80. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } } }
I20260809 04:56:47.073743 9180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:56:47.081907 9066 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.252:39747
I20260809 04:56:47.082082 9298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.252:39747 every 8 connection(s)
I20260809 04:56:47.083901 9066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
I20260809 04:56:47.086246 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 9066
I20260809 04:56:47.090087 9300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:47.099753 9300 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:56:47 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.243.252@KRBTEST.COM
Aug 09 04:56:47 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251406, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.252@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:47.116921 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.088765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.29.243.252:39747)
0809 04:56:47.090181 (+ 1416us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.090201 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.090271 (+ 70us) server_negotiation.cc:407] Connection header received
0809 04:56:47.092624 (+ 2353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.092647 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.093068 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.093620 (+ 552us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:47.095882 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.097686 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.099449 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.100418 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.100601 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.107426 (+ 6825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:47.107436 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:47.107446 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:47.107480 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:47.112001 (+ 4521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.112795 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.112804 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.112815 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.113130 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.113516 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.113521 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.113525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.114255 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:47.114571 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.115130 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.115517 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1125,"thread_start_us":717,"threads_started":1}
I20260809 04:56:47.123793 9309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.102731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52146)
0809 04:56:47.104120 (+ 1389us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.104193 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:47.104480 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:47.104852 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:47.104857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:47.106118 (+ 1261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:47.106601 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.106611 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.108052 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.108058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.110355 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.110370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.111397 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.111407 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.111569 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.112324 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:47.112349 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:47.115723 (+ 3374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:47.120535 (+ 4812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.120546 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.120554 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.121254 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.121786 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.121789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.121792 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.122025 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.122661 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:47.122667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:47.122845 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 04:56:47.123053 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 04:56:47.123322 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1064,"thread_start_us":773,"threads_started":1}
I20260809 04:56:47.123855 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.102899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52146 (local address 127.29.243.254:34625)
0809 04:56:47.103112 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.103118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.104417 (+ 1299us) server_negotiation.cc:407] Connection header received
0809 04:56:47.104596 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.104602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.104693 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.104807 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:47.106748 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.107853 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.110508 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.111092 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.111217 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.115898 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:47.115903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:47.115906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:47.115938 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:47.120405 (+ 4467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.121378 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.121384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.121388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.121637 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.122133 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.122136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.122138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.122515 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:47.122656 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.123199 (+ 543us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.123461 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 04:56:47.125253 9300 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:56:47 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251406, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.252@KRBTEST.COM for oryx/127.29.243.253@KRBTEST.COM
I20260809 04:56:47.141737 9309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.126196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:45079 (local address 127.0.0.1:60764)
0809 04:56:47.126439 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.126468 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:47.126546 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:47.126814 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:47.126817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:47.127309 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:47.127590 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.127599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.128835 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.128839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.130100 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.130111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.130990 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.130994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.131185 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.131859 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:47.131876 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:47.134609 (+ 2733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:47.138769 (+ 4160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.138774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.138779 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.139392 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.139841 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.139846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.139850 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.140037 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.140653 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:47.140658 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:47.140792 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 04:56:47.140946 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 04:56:47.141298 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 04:56:47.141803 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.126363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60764 (local address 127.29.243.253:45079)
0809 04:56:47.126564 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.126570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.126603 (+ 33us) server_negotiation.cc:407] Connection header received
0809 04:56:47.126652 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.126655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.126715 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.126788 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:47.127710 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.128701 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.130212 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.130858 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.130991 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.134853 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:47.134859 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:47.134863 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:47.134903 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:47.138633 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.139519 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.139523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.139526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.139731 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.140152 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.140156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.140158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.140538 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:47.140673 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.141110 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.141394 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:56:47.143370 9300 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:56:47 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251406, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.252@KRBTEST.COM for oryx/127.29.243.252@KRBTEST.COM
I20260809 04:56:47.159909 9309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.144511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:39747 (local address 127.0.0.1:38708)
0809 04:56:47.144726 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.144741 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:47.144848 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:47.145112 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:47.145116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:47.145647 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:47.145931 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.145941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.147134 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.147138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.148312 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.148332 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.149199 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.149207 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.149513 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.150167 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:47.150192 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:47.152807 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:47.157043 (+ 4236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.157047 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.157049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.157599 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.158039 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.158042 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.158044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.158304 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.158909 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:47.158914 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:47.159114 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 04:56:47.159229 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 04:56:47.159452 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 04:56:47.160207 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.144201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38708 (local address 127.29.243.252:39747)
0809 04:56:47.144372 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.144377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.144812 (+ 435us) server_negotiation.cc:407] Connection header received
0809 04:56:47.144943 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.144948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.145004 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.145077 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:47.146056 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.147018 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.148445 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.148978 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.149099 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.153059 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:47.153064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:47.153068 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:47.153099 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:47.156903 (+ 3804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.157712 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.157717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.157719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.157934 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.158414 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.158418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.158421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.158787 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:47.158912 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.159458 (+ 546us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.159694 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:56:47.167107 9300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Bootstrap starting.
I20260809 04:56:47.172086 9300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:47.173986 9300 log.cc:824] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:47.177778 9300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: No bootstrap required, opened a new log
I20260809 04:56:47.189317 9300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } }
I20260809 04:56:47.189781 9300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:47.189975 9300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d05306c85104851ac7150793513ba54, State: Initialized, Role: FOLLOWER
I20260809 04:56:47.190613 9300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } }
I20260809 04:56:47.192214 9312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } } }
I20260809 04:56:47.192893 9312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:56:47.193341 9300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:56:47.209869 9330 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:56:47.210095 9330 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:56:47.213958 9066 master_runner.cc:428] Detected that this master 3d05306c85104851ac7150793513ba54 is joining an existing cluster
I20260809 04:56:47.214138 9066 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.243.252:39747
I20260809 04:56:47.224571 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 8256
Aug 09 04:56:47 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.243.193@KRBTEST.COM
I20260809 04:56:47.264039 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.221559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.29.243.253:45079)
0809 04:56:47.221746 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.221752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.223531 (+ 1779us) server_negotiation.cc:407] Connection header received
0809 04:56:47.224186 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.224193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.224293 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.224373 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:47.227037 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.228529 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.232572 (+ 4043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.233586 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.233695 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.240198 (+ 6503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:47.240206 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:47.240212 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:47.240260 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:47.245420 (+ 5160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.252205 (+ 6785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.252213 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.252217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.252490 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.260191 (+ 7701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.260198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.260201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.260749 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:47.260909 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.263067 (+ 2158us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.263497 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:56:47.265590 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.223585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52154)
0809 04:56:47.232116 (+ 8531us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.232136 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:47.232291 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:47.233020 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:47.233025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:47.234850 (+ 1825us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:47.237240 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.237258 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.240327 (+ 3069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.240334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.245522 (+ 5188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.245538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.247501 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.247512 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.247882 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.248965 (+ 1083us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:47.248993 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:47.251210 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:47.257032 (+ 5822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.257034 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.257036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.261990 (+ 4954us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.262559 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.262564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.262567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.263497 (+ 930us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.264317 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:47.264322 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:47.264642 (+ 320us) client_negotiation.cc:769] Sending connection context
0809 04:56:47.264754 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 04:56:47.265042 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7895,"thread_start_us":7557,"threads_started":1}
I20260809 04:56:47.266070 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.221046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52154 (local address 127.29.243.254:34625)
0809 04:56:47.221490 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.221496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.232223 (+ 10727us) server_negotiation.cc:407] Connection header received
0809 04:56:47.232463 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.232468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.232556 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.232650 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:47.237430 (+ 4780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.239030 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.245766 (+ 6736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.246825 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.246946 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.252326 (+ 5380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:47.252335 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:47.252340 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:47.252387 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:47.256800 (+ 4413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.262113 (+ 5313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.262119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.262122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.262400 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.263626 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.263633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.263637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.264085 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:47.264260 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.265062 (+ 802us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.265304 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260809 04:56:47.270541 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.221605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:45079 (local address 127.0.0.1:60774)
0809 04:56:47.223419 (+ 1814us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.223442 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:47.223994 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:47.224600 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:47.224610 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:47.225898 (+ 1288us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:47.226846 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.226866 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.228864 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.228871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.232405 (+ 3534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.232424 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.234125 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.234131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.234412 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.235381 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:47.235405 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:47.239942 (+ 4537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:47.245618 (+ 5676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.245624 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.245629 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.252032 (+ 6403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.256959 (+ 4927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.256963 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.256965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.260029 (+ 3064us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.260976 (+ 947us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:47.260981 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:47.262708 (+ 1727us) client_negotiation.cc:769] Sending connection context
0809 04:56:47.264252 (+ 1544us) client_negotiation.cc:240] Negotiation successful
0809 04:56:47.267605 (+ 3353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1547,"thread_start_us":661,"threads_started":1}
I20260809 04:56:47.273581 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.226702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36200 (local address 127.29.243.193:45623)
0809 04:56:47.244285 (+ 17583us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.244298 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.244337 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:56:47.244492 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.244505 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.244872 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.247857 (+ 2985us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:47.248410 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.250830 (+ 2420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.253440 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.254738 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.254887 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.262953 (+ 8066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:47.262972 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:47.262994 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:47.263061 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:47.268420 (+ 5359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.269300 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.269308 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.269310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.269506 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.270045 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.270050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.270052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.270624 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:47.272444 (+ 1820us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.272619 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.272965 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17348,"thread_start_us":18312,"threads_started":1}
I20260809 04:56:47.275450 8465 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 04:56:47.287147 9225 tablet.cc:2369] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:56:47.288347 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.230368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:39747 (local address 127.0.0.1:38712)
0809 04:56:47.260552 (+ 30184us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.260574 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:47.260773 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:47.261245 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:47.261249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:47.262962 (+ 1713us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:47.267340 (+ 4378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.267355 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.270058 (+ 2703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.270064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.273203 (+ 3139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.273219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.275318 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.275328 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.275674 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.276573 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:47.276596 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:47.278407 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:47.283897 (+ 5490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.283902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.283906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.284590 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.285089 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.285094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.285097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.285296 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.286133 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:47.286137 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:47.286352 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 04:56:47.287180 (+ 828us) client_negotiation.cc:240] Negotiation successful
0809 04:56:47.287616 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29596,"thread_start_us":29021,"threads_started":1}
I20260809 04:56:47.288674 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.221259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.29.243.252:39747)
0809 04:56:47.221520 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.221527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.260650 (+ 39123us) server_negotiation.cc:407] Connection header received
0809 04:56:47.260902 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.260909 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.260986 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.261062 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:47.268420 (+ 7358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.269731 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.273364 (+ 3633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.274492 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.274591 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.278657 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:47.278664 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:47.278669 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:47.278708 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:47.283739 (+ 5031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.284724 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.284729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.284733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.284945 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.285407 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.285411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.285414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.285889 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:47.286063 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.286796 (+ 733us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.287013 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 04:56:47 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251404, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.254@KRBTEST.COM for oryx/127.29.243.252@KRBTEST.COM
I20260809 04:56:47.310382 9260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.275475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:39747 (local address 127.0.0.1:38718)
0809 04:56:47.275976 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.275997 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:47.276108 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:47.289624 (+ 13516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:47.289630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:47.290249 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:47.290535 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.290552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.292804 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.292811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.294706 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.294725 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.296127 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.296134 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.296405 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.297274 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:47.297300 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:47.301040 (+ 3740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:47.306940 (+ 5900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.306946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.306950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.307711 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.308213 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.308219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.308223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.308483 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.309259 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:47.309265 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:47.309488 (+ 223us) client_negotiation.cc:769] Sending connection context
0809 04:56:47.309613 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 04:56:47.309936 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311}
I20260809 04:56:47.311702 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.276313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38718 (local address 127.29.243.252:39747)
0809 04:56:47.289236 (+ 12923us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.289243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.289262 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:56:47.289358 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.289363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.289458 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.289559 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:47.290715 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.292585 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.295002 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.295909 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.301215 (+ 5306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.301501 (+ 286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:47.301508 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:47.301514 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:47.301545 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:47.306702 (+ 5157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.307868 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.307874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.307878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.308079 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.308627 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.308632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.308636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.309126 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:47.310761 (+ 1635us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.310878 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.311147 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12814,"thread_start_us":21863,"threads_started":1}
I20260809 04:56:47.313421 8162 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.243.252:39747: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3d05306c85104851ac7150793513ba54"
member_type: NON_VOTER
last_known_addr {
host: "127.29.243.252"
port: 39747
}
attrs {
promote: true
}
}
dest_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80"
cas_config_opid_index: 5
I20260809 04:56:47.314839 8162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } }
I20260809 04:56:47.316973 9060 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3d05306c85104851ac7150793513ba54 (127.29.243.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.321811 9060 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.243.252:39747
I20260809 04:56:47.323050 9131 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 LEARNER]: Refusing update from remote peer 1dbf05d2d18743ec936aebcbe9ab5c80: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 04:56:47.323819 9066 master.cc:537] Master@127.29.243.252:39747 shutting down...
I20260809 04:56:47.332988 9353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 04:56:47.336870 9066 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:47.337240 9066 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:47.337432 9066 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: stopping tablet replica
W20260809 04:56:47.338379 8142 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 -> Peer 3d05306c85104851ac7150793513ba54 (127.29.243.252:39747): Couldn't send request to peer 3d05306c85104851ac7150793513ba54. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 04:56:47.345130 9066 master.cc:552] Master@127.29.243.252:39747 shutdown complete.
I20260809 04:56:47.345232 9066 master_runner.cc:305] Clearing existing system tablet
I20260809 04:56:47.345593 9066 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:47.345706 9066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:47.348258 9131 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3d05306c85104851ac7150793513ba54 (127.29.243.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.352762 9180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.353080 9180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:56:47.359935 9354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.360334 9354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:47.360452 9060 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80: attempting to promote NON_VOTER 5be2fa38862348589240908d3ba94459 to VOTER
I20260809 04:56:47.361303 9060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } }
I20260809 04:56:47.363384 9353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: NON_VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: true } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.363799 9353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:47.365038 9260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.364455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:39747 (local address 127.0.0.1:38728)
0809 04:56:47.364702 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.364837 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:39747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260809 04:56:47.366132 9260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.365714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:39747 (local address 127.0.0.1:38742)
0809 04:56:47.365930 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.366006 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:39747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260809 04:56:47.366300 8142 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 -> Peer 3d05306c85104851ac7150793513ba54 (127.29.243.252:39747): Couldn't send request to peer 3d05306c85104851ac7150793513ba54. Status: Network error: Client connection negotiation failed: client connection to 127.29.243.252:39747: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 04:56:47.367493 9131 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 LEARNER]: Refusing update from remote peer 1dbf05d2d18743ec936aebcbe9ab5c80: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 04:56:47.368105 9353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 04:56:47.369343 9066 fs_manager.cc:709] Time spent opening directory manager: real 0.023s user 0.001s sys 0.005s
I20260809 04:56:47.370939 9060 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5be2fa38862348589240908d3ba94459 (127.29.243.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.372215 9130 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5be2fa38862348589240908d3ba94459 (127.29.243.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.375845 9180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.376204 9180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:56:47.379612 9353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.379246 9359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.380481 9359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:47.380663 9353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:47.387698 9362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:47.388219 9066 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:47.388643 9066 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.000s sys 0.009s
I20260809 04:56:47.388854 9066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/wal
uuid: "3d05306c85104851ac7150793513ba54"
format_stamp: "Formatted at 2026-08-09 04:56:46 on dist-test-slave-21b9"
server_key: "6fd8ea40a21603b55334ec79d77a1ea0"
server_key_iv: "d4ac157b6170e105ed3ba21c01cfbb6e"
server_key_version: "encryptionkey@0"
I20260809 04:56:47.393698 9066 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:47.400310 9066 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 04:56:47.400543 9066 log.cc:1196] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 04:56:47.401185 9066 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Deleting consensus metadata
I20260809 04:56:47.401460 9066 master_runner.cc:315] Copying system tablet from 127.29.243.254:34625
I20260809 04:56:47.409998 9066 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: tablet copy: Beginning tablet copy session from remote peer at address 127.29.243.254:34625
I20260809 04:56:47.423915 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 8256
I20260809 04:56:47.433344 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.410903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52156 (local address 127.29.243.254:34625)
0809 04:56:47.411324 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.411332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.414701 (+ 3369us) server_negotiation.cc:407] Connection header received
0809 04:56:47.415006 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.415012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.415091 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.415185 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:47.416626 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.417550 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.419425 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.420274 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.420394 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.423352 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:47.423360 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:47.423365 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:47.423409 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:47.429817 (+ 6408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.430696 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.430702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.430706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.430903 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:47.431421 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:47.431427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:47.431432 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:47.431803 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:47.432075 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.432726 (+ 651us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.432983 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260809 04:56:47.433810 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.410904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52156)
0809 04:56:47.414571 (+ 3667us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.414594 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:47.414791 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:47.415213 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:47.415217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:47.415961 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:47.416475 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.416487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.417745 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.417749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.419284 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.419301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.420435 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.420441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.420708 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.421594 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:47.421651 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:47.423064 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:47.430005 (+ 6941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.430011 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.430016 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.430539 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.431071 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:47.431076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:47.431080 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:47.431262 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:47.432028 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:47.432033 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:47.432357 (+ 324us) client_negotiation.cc:769] Sending connection context
0809 04:56:47.432507 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 04:56:47.433163 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3499,"thread_start_us":3275,"threads_started":1}
I20260809 04:56:47.434598 8189 tablet_copy_service.cc:140] P 1dbf05d2d18743ec936aebcbe9ab5c80: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3d05306c85104851ac7150793513ba54 ({username='oryx', principal='oryx/127.29.243.252@KRBTEST.COM'} at 127.0.0.1:52156)
I20260809 04:56:47.434795 8189 tablet_copy_service.cc:161] P 1dbf05d2d18743ec936aebcbe9ab5c80: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3d05306c85104851ac7150793513ba54 at {username='oryx', principal='oryx/127.29.243.252@KRBTEST.COM'} at 127.0.0.1:52156: session id = 3d05306c85104851ac7150793513ba54-00000000000000000000000000000000
I20260809 04:56:47.438144 8189 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 04:56:47.440284 9066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 04:56:47.447232 9066 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: tablet copy: Starting download of 0 data blocks...
I20260809 04:56:47.447572 9066 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: tablet copy: Starting download of 1 WAL segments...
I20260809 04:56:47.450218 9066 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 04:56:47.451454 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:45623
--local_ip_for_outbound_sockets=127.29.243.193
--tserver_master_addrs=127.29.243.254:34625
--webserver_port=37273
--webserver_interface=127.29.243.193
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab
--principal=oryx/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.243.254:34625,127.29.243.253:45079
--tserver_master_addrs=127.29.243.254:34625,127.29.243.253:45079,127.29.243.252:39747 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:56:47.453975 8189 tablet_copy_service.cc:342] P 1dbf05d2d18743ec936aebcbe9ab5c80: Request end of tablet copy session 3d05306c85104851ac7150793513ba54-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.243.252@KRBTEST.COM'} at 127.0.0.1:52156
I20260809 04:56:47.454110 8189 tablet_copy_service.cc:433] P 1dbf05d2d18743ec936aebcbe9ab5c80: ending tablet copy session 3d05306c85104851ac7150793513ba54-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3d05306c85104851ac7150793513ba54
I20260809 04:56:47.465669 9066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:47.468369 9371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:47.472618 9375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:47.482132 9066 server_base.cc:1034] running on GCE node
W20260809 04:56:47.483100 9377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:47.483682 9066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:47.484488 9066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:47.485744 9066 hybrid_clock.cc:648] HybridClock initialized: now 1786251407485704 us; error 28 us; skew 500 ppm
I20260809 04:56:47.485996 9066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:47 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251407, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:47.488147 9066 init.cc:377] Logged in from keytab as oryx/127.29.243.252@KRBTEST.COM (short username oryx)
I20260809 04:56:47.492201 9066 webserver.cc:460] Webserver started at http://127.29.243.252:33399/ using document root <none> and password file <none>
I20260809 04:56:47.492539 9066 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:47.492647 9066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:47.495694 9066 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:56:47.497624 9391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:47.498019 9066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 04:56:47.498184 9066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/wal
uuid: "3d05306c85104851ac7150793513ba54"
format_stamp: "Formatted at 2026-08-09 04:56:46 on dist-test-slave-21b9"
server_key: "6fd8ea40a21603b55334ec79d77a1ea0"
server_key_iv: "d4ac157b6170e105ed3ba21c01cfbb6e"
server_key_version: "encryptionkey@0"
I20260809 04:56:47.498456 9066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:47.518450 9066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:47.522678 9066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:47.542325 9066 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.252:39747
I20260809 04:56:47.542371 9479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.252:39747 every 8 connection(s)
I20260809 04:56:47.543550 9066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
I20260809 04:56:47.548755 9480 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:56:47.552371 9480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Bootstrap starting.
I20260809 04:56:47.570955 9480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:56:47.571544 9480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Bootstrap complete.
I20260809 04:56:47.572708 9480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } }
I20260809 04:56:47.573035 9480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3d05306c85104851ac7150793513ba54, State: Initialized, Role: LEARNER
I20260809 04:56:47.573305 9480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } }
I20260809 04:56:47.574182 9487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.574486 9480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:47.574714 9487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:56:47.584515 9501 catalog_manager.cc:1259] Loaded cluster ID: c12f80fb769e40f385feb51c1cf0ac40
I20260809 04:56:47.584702 9501 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: loading cluster ID for follower catalog manager: success
I20260809 04:56:47.588593 9501 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: acquiring CA information for follower catalog manager: success
I20260809 04:56:47.591288 9501 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:56:47.841428 9503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.834509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38746 (local address 127.29.243.252:39747)
0809 04:56:47.835395 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:47.835400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:47.835417 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:56:47.835479 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:47.835482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:47.835560 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:47.835660 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:47.836210 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.837122 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.839475 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.840376 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.840521 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.840661 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:47.841057 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 04:56:47.841170 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":581,"threads_started":1}
I20260809 04:56:47.841650 9260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:47.834456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:39747 (local address 127.0.0.1:38746)
0809 04:56:47.834682 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:47.834704 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:47.834856 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:47.835739 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:47.835745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:47.835785 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:47.836038 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.836055 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.837280 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.837288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.839249 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:47.839297 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:47.840561 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:47.840568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:47.840844 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:47.840886 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 04:56:47.840991 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 04:56:47.841180 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 04:56:47.843122 9446 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 04:56:47.843935 9487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dbf05d2d18743ec936aebcbe9ab5c80. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:47.844327 9487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: This master's current role is: LEARNER
W20260809 04:56:47.932940 9370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:47.933355 9370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:47.933562 9370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:47.944052 9370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:47.944257 9370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:47.944541 9370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:56:47.958464 9370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab
--principal=oryx/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:45623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=37273
--tserver_master_addrs=127.29.243.254:34625,127.29.243.253:45079,127.29.243.252:39747
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:47.960171 9370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:47.961809 9370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:47.980672 9509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:47.981324 9512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:47.981428 9514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:47.982697 9370 server_base.cc:1034] running on GCE node
I20260809 04:56:47.983670 9370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:47.984941 9370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:47.986225 9370 hybrid_clock.cc:648] HybridClock initialized: now 1786251407986111 us; error 50 us; skew 500 ppm
I20260809 04:56:47.986651 9370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:47 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251407, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:47.991544 9370 init.cc:377] Logged in from keytab as oryx/127.29.243.193@KRBTEST.COM (short username oryx)
I20260809 04:56:47.993506 9370 webserver.cc:460] Webserver started at http://127.29.243.193:37273/ using document root <none> and password file <none>
I20260809 04:56:47.994381 9370 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:47.994621 9370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:48.001893 9370 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.005s
I20260809 04:56:48.005661 9526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:48.007110 9370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:48.007350 9370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "97b24491f3874d1fa98090dc7e4ee226"
format_stamp: "Formatted at 2026-08-09 04:56:44 on dist-test-slave-21b9"
server_key: "08b4d7661f26c0dfdc5639cc7586be7c"
server_key_iv: "4254f25476a05074135cefb4882f3c5a"
server_key_version: "encryptionkey@0"
I20260809 04:56:48.008805 9370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:48.025909 9370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:48.031677 9370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:48.032052 9370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:48.033655 9370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:48.035962 9370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:48.036149 9370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:48.036327 9370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:48.036459 9370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:48.121434 9370 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:45623
I20260809 04:56:48.121644 9708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:45623 every 8 connection(s)
I20260809 04:56:48.123498 9370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:56:48.128851 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 9370
I20260809 04:56:48.129267 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 8482
Aug 09 04:56:48 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.243.194@KRBTEST.COM
Aug 09 04:56:48 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251407, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.253@KRBTEST.COM
Aug 09 04:56:48 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251407, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:48.188814 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.131227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58928 (local address 127.29.243.194:41757)
0809 04:56:48.155783 (+ 24556us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:48.155794 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:48.158045 (+ 2251us) server_negotiation.cc:407] Connection header received
0809 04:56:48.158226 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:48.158236 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:48.158524 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:48.158719 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:48.162158 (+ 3439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.164266 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.166675 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.167779 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.167906 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.177283 (+ 9377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:48.177302 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:48.177322 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:48.177369 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:48.182906 (+ 5537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.184343 (+ 1437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.184356 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.184360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.184717 (+ 357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.185269 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.185274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.185278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.185900 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:48.186268 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:48.187741 (+ 1473us) server_negotiation.cc:299] Negotiation successful
0809 04:56:48.188062 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24289,"thread_start_us":24488,"threads_started":1}
I20260809 04:56:48.190665 8673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:48.191428 9716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.127560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:48079 (local address 127.29.243.253:45079)
0809 04:56:48.142910 (+ 15350us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:48.142915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:48.157796 (+ 14881us) server_negotiation.cc:407] Connection header received
0809 04:56:48.162602 (+ 4806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:48.162608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:48.162751 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:48.162837 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:48.166910 (+ 4073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.168364 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.172613 (+ 4249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.173645 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.173752 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.181466 (+ 7714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:48.181472 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:48.181477 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:48.181522 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:48.186553 (+ 5031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.187727 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.187733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.187736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.187919 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.188560 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.188565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.188568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.189310 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:48.189440 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:48.190605 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0809 04:56:48.190937 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15233,"thread_start_us":15365,"threads_started":1}
I20260809 04:56:48.191906 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.127579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:56003 (local address 127.29.243.254:34625)
0809 04:56:48.140424 (+ 12845us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:48.140431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:48.160735 (+ 20304us) server_negotiation.cc:407] Connection header received
0809 04:56:48.162911 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:48.162917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:48.163004 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:48.163091 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:48.167100 (+ 4009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.168419 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.173704 (+ 5285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.174639 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.174745 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.180947 (+ 6202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:48.180956 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:48.180961 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:48.181004 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:48.186389 (+ 5385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.187827 (+ 1438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.187833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.187836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.188025 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.188591 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.188595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.188598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.190187 (+ 1589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:48.190312 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:48.190979 (+ 667us) server_negotiation.cc:299] Negotiation successful
0809 04:56:48.191240 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12694,"thread_start_us":12862,"threads_started":1}
I20260809 04:56:48.193084 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.130126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.193:56003)
0809 04:56:48.160603 (+ 30477us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:48.160624 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:48.162224 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:48.163547 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:48.163560 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:48.165400 (+ 1840us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:48.166896 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.166912 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.168723 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.168728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.173518 (+ 4790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.173535 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.175393 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.175404 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.175685 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.176577 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:48.176602 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:48.180673 (+ 4071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:48.186658 (+ 5985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.186667 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.186684 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.187705 (+ 1021us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.188156 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.188161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.188164 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.188461 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.190402 (+ 1941us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:48.190406 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:48.190621 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 04:56:48.190821 (+ 200us) client_negotiation.cc:240] Negotiation successful
0809 04:56:48.191401 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30312,"mutex_wait_us":208,"thread_start_us":26388,"threads_started":1}
I20260809 04:56:48.193334 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.128504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:45079 (local address 127.29.243.193:48079)
0809 04:56:48.157474 (+ 28970us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:48.157573 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:48.162210 (+ 4637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:48.163549 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:48.163562 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:48.165218 (+ 1656us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:48.166632 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.166674 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.169190 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.169197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.172420 (+ 3223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.172437 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.174277 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.174288 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.174704 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.175589 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:48.175625 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:48.181258 (+ 5633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:48.186684 (+ 5426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.186689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.186692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.187573 (+ 881us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.188081 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.188085 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.188089 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.188422 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.189580 (+ 1158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:48.189587 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:48.190100 (+ 513us) client_negotiation.cc:769] Sending connection context
0809 04:56:48.191905 (+ 1805us) client_negotiation.cc:240] Negotiation successful
0809 04:56:48.192551 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28368,"mutex_wait_us":167,"spinlock_wait_cycles":45824,"thread_start_us":31303,"threads_started":1}
I20260809 04:56:48.195076 9712 heartbeater.cc:347] Connected to a master server at 127.29.243.253:45079
I20260809 04:56:48.195163 9711 heartbeater.cc:347] Connected to a master server at 127.29.243.254:34625
I20260809 04:56:48.195499 9712 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:48.195523 9711 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:48.197121 9711 heartbeater.cc:511] Master 127.29.243.254:34625 requested a full tablet report, sending...
I20260809 04:56:48.198385 9712 heartbeater.cc:511] Master 127.29.243.253:45079 requested a full tablet report, sending...
I20260809 04:56:48.199726 8162 ts_manager.cc:194] Re-registered known tserver with Master: 97b24491f3874d1fa98090dc7e4ee226 (127.29.243.193:45623)
I20260809 04:56:48.200793 9115 ts_manager.cc:194] Registered new tserver with Master: 97b24491f3874d1fa98090dc7e4ee226 (127.29.243.193:45623)
Aug 09 04:56:48 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251407, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.252@KRBTEST.COM
I20260809 04:56:48.201231 8162 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:56003
I20260809 04:56:48.213344 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.129193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:39747 (local address 127.29.243.193:58423)
0809 04:56:48.159743 (+ 30550us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:48.159764 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:48.162210 (+ 2446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:48.163549 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:48.163562 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:48.165303 (+ 1741us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:48.166829 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.166845 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.168704 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.168711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.187796 (+ 19085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.187822 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.189592 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.189599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.189784 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.196646 (+ 6862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:48.196668 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:48.203154 (+ 6486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:48.208788 (+ 5634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.208793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.208798 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.209932 (+ 1134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.210631 (+ 699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.210635 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.210639 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.210918 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.211896 (+ 978us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:48.211902 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:48.212131 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 04:56:48.212335 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 04:56:48.212708 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30349,"mutex_wait_us":277,"thread_start_us":29934,"threads_started":1}
I20260809 04:56:48.213547 9503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.127828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:58423 (local address 127.29.243.252:39747)
0809 04:56:48.129923 (+ 2095us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:48.129931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:48.159890 (+ 29959us) server_negotiation.cc:407] Connection header received
0809 04:56:48.162601 (+ 2711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:48.162608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:48.162709 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:48.162788 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:48.167022 (+ 4234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.168428 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.188021 (+ 19593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.189016 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.189160 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.203533 (+ 14373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:48.203542 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:48.203546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:48.203592 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:48.208600 (+ 5008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.210174 (+ 1574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.210184 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.210187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.210488 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.211069 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.211075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.211079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.211750 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:48.211926 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:48.212531 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 04:56:48.212932 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1936}
I20260809 04:56:48.214951 9710 heartbeater.cc:347] Connected to a master server at 127.29.243.252:39747
I20260809 04:56:48.215185 9710 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:48.216014 9710 heartbeater.cc:511] Master 127.29.243.252:39747 requested a full tablet report, sending...
I20260809 04:56:48.221383 9425 ts_manager.cc:194] Registered new tserver with Master: 97b24491f3874d1fa98090dc7e4ee226 (127.29.243.193:45623)
I20260809 04:56:48.281870 9353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: NON_VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: true } } }
I20260809 04:56:48.282276 9353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:48.284125 9731 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80: attempting to promote NON_VOTER 3d05306c85104851ac7150793513ba54 to VOTER
I20260809 04:56:48.285046 9731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } }
I20260809 04:56:48.287621 9446 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 1 LEARNER]: Refusing update from remote peer 1dbf05d2d18743ec936aebcbe9ab5c80: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 04:56:48.287768 9130 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 FOLLOWER]: Refusing update from remote peer 1dbf05d2d18743ec936aebcbe9ab5c80: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 04:56:48.288342 9728 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 04:56:48.288923 9353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 04:56:48.291610 9728 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3d05306c85104851ac7150793513ba54 (127.29.243.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:48.292620 9130 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3d05306c85104851ac7150793513ba54 (127.29.243.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:48.295663 9353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:48.295976 9353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:48.296288 9353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:48.296675 9353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:48.297298 9735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:48.297904 9735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:56:48.300000 9728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:48.300268 9728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:48.301060 9446 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3d05306c85104851ac7150793513ba54 (127.29.243.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:48.304132 9487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:48.304382 9487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:56:48.314828 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 8482
I20260809 04:56:48.340783 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:41757
--local_ip_for_outbound_sockets=127.29.243.194
--tserver_master_addrs=127.29.243.254:34625
--webserver_port=37773
--webserver_interface=127.29.243.194
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab
--principal=oryx/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.243.254:34625,127.29.243.253:45079
--tserver_master_addrs=127.29.243.254:34625,127.29.243.253:45079,127.29.243.252:39747 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:48.686388 9736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:48.686657 9736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:48.686774 9736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:48.694018 9736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:48.694126 9736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:48.694291 9736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:56:48.704492 9736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab
--principal=oryx/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:41757
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=37773
--tserver_master_addrs=127.29.243.254:34625,127.29.243.253:45079,127.29.243.252:39747
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:48.705852 9736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:48.707259 9736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:48.720680 9750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:48.720690 9757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:48.721020 9754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:48.721797 9736 server_base.cc:1034] running on GCE node
I20260809 04:56:48.722370 9736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:48.723465 9736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:48.724731 9736 hybrid_clock.cc:648] HybridClock initialized: now 1786251408724668 us; error 47 us; skew 500 ppm
I20260809 04:56:48.725036 9736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:48 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251408, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:48.728304 9736 init.cc:377] Logged in from keytab as oryx/127.29.243.194@KRBTEST.COM (short username oryx)
I20260809 04:56:48.729717 9736 webserver.cc:460] Webserver started at http://127.29.243.194:37773/ using document root <none> and password file <none>
I20260809 04:56:48.730388 9736 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:48.730525 9736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:48.736184 9736 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:56:48.739655 9769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:48.741219 9736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:56:48.741389 9736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "7ff56221a0be4ba392f866e0ea37cfa6"
format_stamp: "Formatted at 2026-08-09 04:56:45 on dist-test-slave-21b9"
server_key: "2e1deab75d59d657c88ec4ca17fb2ebd"
server_key_iv: "9ff870df1adb7bad63a3983916f45201"
server_key_version: "encryptionkey@0"
I20260809 04:56:48.742813 9736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:48.777810 9736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:48.782239 9736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:48.782534 9736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:48.783576 9736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:48.785001 9736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:48.785095 9736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:48.785192 9736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:48.785247 9736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:48.817992 9736 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:41757
I20260809 04:56:48.818055 9957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:41757 every 8 connection(s)
I20260809 04:56:48.819695 9736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:56:48.825680 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 9736
I20260809 04:56:48.825946 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 8687
Aug 09 04:56:48 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251408, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.252@KRBTEST.COM
Aug 09 04:56:48 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251408, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
Aug 09 04:56:48 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251408, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.253@KRBTEST.COM
Aug 09 04:56:48 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.243.195@KRBTEST.COM
I20260809 04:56:48.860304 9967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.822777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:58385 (local address 127.29.243.252:39747)
0809 04:56:48.823830 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:48.823836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:48.835407 (+ 11571us) server_negotiation.cc:407] Connection header received
0809 04:56:48.835965 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:48.835972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:48.836064 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:48.836159 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:48.839669 (+ 3510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.840719 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.843374 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.844162 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.844298 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.850532 (+ 6234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:48.850538 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:48.850542 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:48.850581 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:48.855251 (+ 4670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.856695 (+ 1444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.856701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.856705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.856897 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.857550 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.857555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.857558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.857998 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:48.858130 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:48.859300 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0809 04:56:48.859552 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929,"thread_start_us":826,"threads_started":1}
I20260809 04:56:48.860303 9966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.822627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:44525 (local address 127.29.243.254:34625)
0809 04:56:48.823094 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:48.823100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:48.835918 (+ 12818us) server_negotiation.cc:407] Connection header received
0809 04:56:48.836352 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:48.836358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:48.836447 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:48.836527 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:48.839325 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.840476 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.845222 (+ 4746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.845805 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.845918 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.850468 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:48.850475 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:48.850479 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:48.850523 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:48.855470 (+ 4947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.856695 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.856701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.856705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.856913 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.857549 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.857555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.857558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.858015 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:48.858129 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:48.859326 (+ 1197us) server_negotiation.cc:299] Negotiation successful
0809 04:56:48.859582 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":784,"threads_started":1}
I20260809 04:56:48.862304 9970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.823339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.194:44525)
0809 04:56:48.835791 (+ 12452us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:48.835811 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:48.836189 (+ 378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:48.836745 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:48.836750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:48.838561 (+ 1811us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:48.839068 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.839098 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.840670 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.840675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.845057 (+ 4382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.845072 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.846202 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.846208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.846406 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.847126 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:48.847147 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:48.850222 (+ 3075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:48.855630 (+ 5408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.855634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.855637 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.856560 (+ 923us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.857050 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.857055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.857059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.857415 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.858266 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:48.858274 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:48.858759 (+ 485us) client_negotiation.cc:769] Sending connection context
0809 04:56:48.860587 (+ 1828us) client_negotiation.cc:240] Negotiation successful
0809 04:56:48.860900 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12288,"mutex_wait_us":3299,"spinlock_wait_cycles":62336,"thread_start_us":354,"threads_started":1}
I20260809 04:56:48.862370 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.823637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:39747 (local address 127.29.243.194:58385)
0809 04:56:48.835296 (+ 11659us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:48.835317 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:48.835783 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:48.836733 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:48.836746 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:48.839022 (+ 2276us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:48.839472 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.839485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.840967 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.840971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.843176 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.843191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.844404 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.844410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.844600 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.845267 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:48.845300 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:48.850214 (+ 4914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:48.855461 (+ 5247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.855470 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.855486 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.856560 (+ 1074us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.857050 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.857055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.857059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.857415 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.858266 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:48.858274 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:48.858759 (+ 485us) client_negotiation.cc:769] Sending connection context
0809 04:56:48.860587 (+ 1828us) client_negotiation.cc:240] Negotiation successful
0809 04:56:48.861034 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11495,"mutex_wait_us":291,"spinlock_wait_cycles":11392,"thread_start_us":12304,"threads_started":1}
I20260809 04:56:48.862880 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.823909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:58363 (local address 127.29.243.253:45079)
0809 04:56:48.824332 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:48.824338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:48.825185 (+ 847us) server_negotiation.cc:407] Connection header received
0809 04:56:48.836291 (+ 11106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:48.836297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:48.836412 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:48.836496 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:48.839350 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.840476 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.844493 (+ 4017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.845107 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.845217 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.851408 (+ 6191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:48.851415 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:48.851420 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:48.851462 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:48.856767 (+ 5305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.857689 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.857694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.857698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.857889 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.858479 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.858484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.858487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.859017 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:48.859156 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:48.862081 (+ 2925us) server_negotiation.cc:299] Negotiation successful
0809 04:56:48.862336 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":218,"threads_started":1}
I20260809 04:56:48.863346 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.823338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:45079 (local address 127.29.243.194:58363)
0809 04:56:48.825026 (+ 1688us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:48.825073 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:48.833979 (+ 8906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:48.836742 (+ 2763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:48.836747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:48.837869 (+ 1122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:48.839079 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.839103 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.840668 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.840673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.843176 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:48.843191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.845448 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.845454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:48.845633 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.846210 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:48.846231 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:48.851095 (+ 4864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:48.856950 (+ 5855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.856955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.856958 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.857546 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.858031 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:48.858036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:48.858039 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:48.858318 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:48.859222 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:48.859226 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:48.859494 (+ 268us) client_negotiation.cc:769] Sending connection context
0809 04:56:48.859661 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 04:56:48.860416 (+ 755us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1381,"mutex_wait_us":289,"thread_start_us":11938,"threads_started":1}
I20260809 04:56:48.864892 9961 heartbeater.cc:347] Connected to a master server at 127.29.243.254:34625
I20260809 04:56:48.864992 9963 heartbeater.cc:347] Connected to a master server at 127.29.243.253:45079
I20260809 04:56:48.865067 9960 heartbeater.cc:347] Connected to a master server at 127.29.243.252:39747
I20260809 04:56:48.865321 9963 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:48.865384 9960 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:48.865386 9961 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:48.867048 9961 heartbeater.cc:511] Master 127.29.243.254:34625 requested a full tablet report, sending...
I20260809 04:56:48.867154 9963 heartbeater.cc:511] Master 127.29.243.253:45079 requested a full tablet report, sending...
I20260809 04:56:48.867213 9960 heartbeater.cc:511] Master 127.29.243.252:39747 requested a full tablet report, sending...
I20260809 04:56:48.869167 8162 ts_manager.cc:194] Re-registered known tserver with Master: 7ff56221a0be4ba392f866e0ea37cfa6 (127.29.243.194:41757)
I20260809 04:56:48.869529 9425 ts_manager.cc:194] Registered new tserver with Master: 7ff56221a0be4ba392f866e0ea37cfa6 (127.29.243.194:41757)
I20260809 04:56:48.870337 8162 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:44525
I20260809 04:56:48.871263 9115 ts_manager.cc:194] Registered new tserver with Master: 7ff56221a0be4ba392f866e0ea37cfa6 (127.29.243.194:41757)
I20260809 04:56:48.874801 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:48.826898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47968 (local address 127.29.243.195:37453)
0809 04:56:48.842159 (+ 15261us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:48.842167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:48.842197 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:56:48.842261 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:48.842269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:48.842617 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:48.842825 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:48.845133 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.847160 (+ 2027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.849231 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:48.850406 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:48.850536 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:48.859161 (+ 8625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:48.859178 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:48.859197 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:48.859292 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:48.870683 (+ 11391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.871997 (+ 1314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.872006 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.872008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.872223 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:48.872838 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:48.872844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:48.872847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:48.873302 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:48.873584 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:48.873978 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 04:56:48.874223 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15083,"thread_start_us":16218,"threads_started":1}
I20260809 04:56:48.876350 8884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:48.960518 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 8687
W20260809 04:56:48.983093 8140 connection.cc:535] server connection from 127.29.243.195:47611 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 04:56:48.983771 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:37453
--local_ip_for_outbound_sockets=127.29.243.195
--tserver_master_addrs=127.29.243.254:34625
--webserver_port=39131
--webserver_interface=127.29.243.195
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab
--principal=oryx/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.243.254:34625,127.29.243.253:45079
--tserver_master_addrs=127.29.243.254:34625,127.29.243.253:45079,127.29.243.252:39747 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:56:49.204603 9711 heartbeater.cc:503] Master 127.29.243.254:34625 was elected leader, sending a full tablet report...
W20260809 04:56:49.325084 9979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:49.325378 9979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:49.325515 9979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:49.333214 9979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:49.333333 9979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:49.333488 9979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:56:49.343099 9979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab
--principal=oryx/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:37453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=39131
--tserver_master_addrs=127.29.243.254:34625,127.29.243.253:45079,127.29.243.252:39747
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:49.344431 9979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:49.345758 9979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:49.358878 9986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:49.358851 9989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:49.359055 9987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:49.360669 9979 server_base.cc:1034] running on GCE node
I20260809 04:56:49.361316 9979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:49.362241 9979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:49.363435 9979 hybrid_clock.cc:648] HybridClock initialized: now 1786251409363371 us; error 60 us; skew 500 ppm
I20260809 04:56:49.363770 9979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:49 dist-test-slave-21b9 krb5kdc[8097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251409, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:49.366887 9979 init.cc:377] Logged in from keytab as oryx/127.29.243.195@KRBTEST.COM (short username oryx)
I20260809 04:56:49.368237 9979 webserver.cc:460] Webserver started at http://127.29.243.195:39131/ using document root <none> and password file <none>
I20260809 04:56:49.368909 9979 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:49.369024 9979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:49.374233 9979 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:56:49.377146 10009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:49.378531 9979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:56:49.378666 9979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "0490a08d31ba4a588b0a4129ab632520"
format_stamp: "Formatted at 2026-08-09 04:56:45 on dist-test-slave-21b9"
server_key: "45fda7d624ec6172b9d99e4bf3c9044e"
server_key_iv: "8b22a27e6cc51a8456045876859d966a"
server_key_version: "encryptionkey@0"
I20260809 04:56:49.379630 9979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:49.391640 9979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:49.395339 9979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:49.395624 9979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:49.396497 9979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:56:49.397866 9979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:49.397950 9979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:49.398028 9979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:49.398082 9979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:49.429154 9979 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:37453
I20260809 04:56:49.429247 10181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:37453 every 8 connection(s)
I20260809 04:56:49.430613 9979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:56:49.437850 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 9979
Aug 09 04:56:49 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251409, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.252@KRBTEST.COM
Aug 09 04:56:49 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251409, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.253@KRBTEST.COM
I20260809 04:56:49.479333 10196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.434274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:39747 (local address 127.29.243.195:47189)
0809 04:56:49.447845 (+ 13571us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:49.447867 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:49.450477 (+ 2610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:49.451842 (+ 1365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:49.451847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:49.452422 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:49.453515 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.453546 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.456629 (+ 3083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.456636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.458895 (+ 2259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.458913 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.460364 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.460372 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.460699 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.461542 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:49.461573 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:49.465432 (+ 3859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:49.471010 (+ 5578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:49.471022 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:49.471046 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:49.473681 (+ 2635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:49.474306 (+ 625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:49.474311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:49.474314 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:49.474672 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:49.475669 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:49.475679 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:49.476281 (+ 602us) client_negotiation.cc:769] Sending connection context
0809 04:56:49.476685 (+ 404us) client_negotiation.cc:240] Negotiation successful
0809 04:56:49.477328 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13430,"mutex_wait_us":6,"spinlock_wait_cycles":40192,"thread_start_us":13107,"threads_started":1}
I20260809 04:56:49.480703 10187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.434012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:47189 (local address 127.29.243.252:39747)
0809 04:56:49.448361 (+ 14349us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:49.448365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:49.448383 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:56:49.451414 (+ 3031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:49.451420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:49.451520 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:49.451636 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:49.455165 (+ 3529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.456411 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.459116 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.459882 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.460005 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.465708 (+ 5703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:49.465714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:49.465718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:49.465762 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:49.470820 (+ 5058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.473840 (+ 3020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.473847 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.473852 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.474156 (+ 304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.474828 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.474833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.474836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.475366 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:49.475560 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:49.479933 (+ 4373us) server_negotiation.cc:299] Negotiation successful
0809 04:56:49.480167 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14244,"thread_start_us":14125,"threads_started":1}
I20260809 04:56:49.482528 10182 heartbeater.cc:347] Connected to a master server at 127.29.243.252:39747
Aug 09 04:56:49 dist-test-slave-21b9 krb5kdc[8097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251409, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:49.482862 10182 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:49.484252 10182 heartbeater.cc:511] Master 127.29.243.252:39747 requested a full tablet report, sending...
I20260809 04:56:49.486445 10190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.434580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:42961 (local address 127.29.243.253:45079)
0809 04:56:49.445513 (+ 10933us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:49.445521 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:49.448481 (+ 2960us) server_negotiation.cc:407] Connection header received
0809 04:56:49.450831 (+ 2350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:49.450836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:49.450914 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:49.451013 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:49.453743 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.454851 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.464554 (+ 9703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.465519 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.465655 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.474276 (+ 8621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:49.474285 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:49.474290 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:49.474335 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:49.480174 (+ 5839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.483522 (+ 3348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.483528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.483531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.483766 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.484378 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.484395 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.484398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.484842 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:49.484981 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:49.485590 (+ 609us) server_negotiation.cc:299] Negotiation successful
0809 04:56:49.485822 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10767,"thread_start_us":10946,"threads_started":1}
I20260809 04:56:49.486838 9425 ts_manager.cc:194] Registered new tserver with Master: 0490a08d31ba4a588b0a4129ab632520 (127.29.243.195:37453)
I20260809 04:56:49.487866 10189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.434233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:45079 (local address 127.29.243.195:42961)
0809 04:56:49.448363 (+ 14130us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:49.448381 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:49.450477 (+ 2096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:49.451455 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:49.451468 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:49.452955 (+ 1487us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:49.453547 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.453557 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.461073 (+ 7516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.461078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.462612 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.462628 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.466083 (+ 3455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.466091 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.466331 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.467732 (+ 1401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:49.467756 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:49.472880 (+ 5124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:49.481450 (+ 8570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:49.481455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:49.481459 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:49.482355 (+ 896us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:49.483937 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:49.483942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:49.483946 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:49.484227 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:49.485076 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:49.485081 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:49.485262 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 04:56:49.486855 (+ 1593us) client_negotiation.cc:240] Negotiation successful
0809 04:56:49.487207 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13979,"mutex_wait_us":501,"thread_start_us":12271,"threads_started":1}
I20260809 04:56:49.488866 10184 heartbeater.cc:347] Connected to a master server at 127.29.243.253:45079
I20260809 04:56:49.489051 10184 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:49.489822 10184 heartbeater.cc:511] Master 127.29.243.253:45079 requested a full tablet report, sending...
I20260809 04:56:49.492882 9115 ts_manager.cc:194] Registered new tserver with Master: 0490a08d31ba4a588b0a4129ab632520 (127.29.243.195:37453)
I20260809 04:56:49.501600 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.433525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:33245 (local address 127.29.243.254:34625)
0809 04:56:49.445970 (+ 12445us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:49.445976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:49.447160 (+ 1184us) server_negotiation.cc:407] Connection header received
0809 04:56:49.450816 (+ 3656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:49.450822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:49.450911 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:49.451146 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:49.453701 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.454848 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.471561 (+ 16713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.472733 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.472865 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.492616 (+ 19751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:49.492625 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:49.492630 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:49.492677 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:49.498220 (+ 5543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.498953 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.498959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.498963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.499199 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.499752 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.499757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.499760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.500216 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:49.500373 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:49.500832 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 04:56:49.501053 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12308,"thread_start_us":12198,"threads_started":1}
I20260809 04:56:49.501746 10197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.434233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.195:33245)
0809 04:56:49.446928 (+ 12695us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:49.447016 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:49.450473 (+ 3457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:49.451460 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:49.451470 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:49.452422 (+ 952us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:49.453514 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.453550 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.469832 (+ 16282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.469839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.471377 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.471394 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.477184 (+ 5790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.477191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.477438 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.479828 (+ 2390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:49.479852 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:49.492298 (+ 12446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:49.498368 (+ 6070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:49.498374 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:49.498378 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:49.498817 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:49.499342 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:49.499347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:49.499351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:49.499627 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:49.500340 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:49.500345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:49.500516 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 04:56:49.500665 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 04:56:49.501033 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12332,"mutex_wait_us":205,"spinlock_wait_cycles":4864,"thread_start_us":13748,"threads_started":1}
I20260809 04:56:49.502858 10183 heartbeater.cc:347] Connected to a master server at 127.29.243.254:34625
I20260809 04:56:49.503106 10183 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:49.504055 10183 heartbeater.cc:511] Master 127.29.243.254:34625 requested a full tablet report, sending...
I20260809 04:56:49.505388 8162 ts_manager.cc:194] Re-registered known tserver with Master: 0490a08d31ba4a588b0a4129ab632520 (127.29.243.195:37453)
I20260809 04:56:49.506275 8162 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:33245
I20260809 04:56:49.525269 10190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.454733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60782 (local address 127.29.243.253:45079)
0809 04:56:49.486727 (+ 31994us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:49.486733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:49.497151 (+ 10418us) server_negotiation.cc:407] Connection header received
0809 04:56:49.497517 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:49.497523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:49.497641 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:49.497719 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:49.507453 (+ 9734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.508861 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.510995 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.512054 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.512195 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.516240 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:49.516248 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:49.516254 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:49.516301 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:49.521197 (+ 4896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.522274 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.522282 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.522285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.522497 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.523010 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.523015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.523019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.523545 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:49.523875 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:49.524365 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 04:56:49.524624 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31860,"thread_start_us":40917,"threads_started":1}
I20260809 04:56:49.526095 10207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.454919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38758 (local address 127.29.243.252:39747)
0809 04:56:49.475935 (+ 21016us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:49.475941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:49.497797 (+ 21856us) server_negotiation.cc:407] Connection header received
0809 04:56:49.497963 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:49.497969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:49.498048 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:49.498204 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:49.503955 (+ 5751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.505770 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.508489 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.509431 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.509536 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.516497 (+ 6961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:49.516508 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:49.516513 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:49.516560 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:49.521706 (+ 5146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.522772 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.522777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.522781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.523055 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.523830 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.523836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.523839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.524258 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:49.524535 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:49.525197 (+ 662us) server_negotiation.cc:299] Negotiation successful
0809 04:56:49.525494 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20859,"thread_start_us":20997,"threads_started":1}
I20260809 04:56:49.526327 10209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.455163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52164 (local address 127.29.243.254:34625)
0809 04:56:49.483311 (+ 28148us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:49.483317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:49.497970 (+ 14653us) server_negotiation.cc:407] Connection header received
0809 04:56:49.498035 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:49.498039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:49.498172 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:49.498271 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:49.504032 (+ 5761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.505770 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.509166 (+ 3396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.510183 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.510279 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.516333 (+ 6054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:49.516341 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:49.516345 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:49.516381 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:49.521817 (+ 5436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.522852 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.522857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.522860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.523056 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:49.523592 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:49.523598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:49.523602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:49.524048 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:49.524316 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:49.525043 (+ 727us) server_negotiation.cc:299] Negotiation successful
0809 04:56:49.525840 (+ 797us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27983,"thread_start_us":28158,"threads_started":1}
I20260809 04:56:49.531472 8162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:49.543193 8162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:49.596319 10229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.571888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:41757 (local address 127.0.0.1:58942)
0809 04:56:49.584366 (+ 12478us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:49.584391 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:49.584577 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:49.585797 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:49.585804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:49.585841 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:49.586782 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.586798 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.589161 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.589167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.592380 (+ 3213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.592397 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.595231 (+ 2834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.595239 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.595508 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.595543 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 04:56:49.595636 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 04:56:49.595778 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12246,"thread_start_us":14735,"threads_started":1}
I20260809 04:56:49.597532 10228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.572231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58942 (local address 127.29.243.194:41757)
0809 04:56:49.584948 (+ 12717us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:49.584959 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:49.584995 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:56:49.585121 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:49.585132 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:49.585483 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:49.585784 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:49.587060 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.588907 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.592582 (+ 3675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.594559 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.594762 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.594963 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:49.596926 (+ 1963us) server_negotiation.cc:299] Negotiation successful
0809 04:56:49.597160 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12464,"thread_start_us":13734,"threads_started":1}
I20260809 04:56:49.599676 10227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.572740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:37453 (local address 127.0.0.1:47976)
0809 04:56:49.587132 (+ 14392us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:49.587154 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:49.587341 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:49.588570 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:49.588575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:49.588600 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:49.589554 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.589569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.592512 (+ 2943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.592517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.595917 (+ 3400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.595937 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.598555 (+ 2618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.598563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.598832 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.598858 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:56:49.598943 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 04:56:49.599073 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14228,"thread_start_us":11941,"threads_started":1}
I20260809 04:56:49.599891 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.573710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47976 (local address 127.29.243.195:37453)
0809 04:56:49.586009 (+ 12299us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:49.586017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:49.587472 (+ 1455us) server_negotiation.cc:407] Connection header received
0809 04:56:49.587601 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:49.587608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:49.587890 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:49.588105 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:49.589888 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.592308 (+ 2420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.596135 (+ 3827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.598265 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.598455 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.598655 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:49.599023 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 04:56:49.599252 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12033,"thread_start_us":11567,"threads_started":1}
I20260809 04:56:49.604527 10069 tablet_service.cc:1467] Processing CreateTablet for tablet b3c6fd0b266a4fe885f1802a566aed06 (DEFAULT_TABLE table=test-table [id=922aaa2e39cd4dfab8c21567bc535005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:49.606456 10069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c6fd0b266a4fe885f1802a566aed06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:49.609242 9855 tablet_service.cc:1467] Processing CreateTablet for tablet b3c6fd0b266a4fe885f1802a566aed06 (DEFAULT_TABLE table=test-table [id=922aaa2e39cd4dfab8c21567bc535005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:49.610940 9855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c6fd0b266a4fe885f1802a566aed06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:49.615114 10229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.587395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45623 (local address 127.0.0.1:36202)
0809 04:56:49.596621 (+ 9226us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:49.596641 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:49.596740 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:49.605801 (+ 9061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:49.605806 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:49.605834 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:49.606193 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.606208 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.608518 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.608524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.611093 (+ 2569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.611110 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.614053 (+ 2943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.614059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.614360 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.614400 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 04:56:49.614482 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 04:56:49.614693 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9124,"thread_start_us":16357,"threads_started":1}
I20260809 04:56:49.616535 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.587341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36202 (local address 127.29.243.193:45623)
0809 04:56:49.604747 (+ 17406us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:49.604755 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:49.604788 (+ 33us) server_negotiation.cc:407] Connection header received
0809 04:56:49.604901 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:49.604909 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:49.605238 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:49.605485 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:49.606395 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.608323 (+ 1928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.611333 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.613415 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.615521 (+ 2106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.615722 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:49.615799 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 04:56:49.616005 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17166,"thread_start_us":16986,"threads_started":1}
I20260809 04:56:49.622159 9598 tablet_service.cc:1467] Processing CreateTablet for tablet b3c6fd0b266a4fe885f1802a566aed06 (DEFAULT_TABLE table=test-table [id=922aaa2e39cd4dfab8c21567bc535005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:49.624094 9598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c6fd0b266a4fe885f1802a566aed06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:49.627094 10234 tablet_bootstrap.cc:492] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: Bootstrap starting.
I20260809 04:56:49.631950 10235 tablet_bootstrap.cc:492] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: Bootstrap starting.
I20260809 04:56:49.634028 10234 tablet_bootstrap.cc:654] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:49.636626 10234 log.cc:824] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:49.637981 10235 tablet_bootstrap.cc:654] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:49.640163 10235 log.cc:824] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:49.641808 10234 tablet_bootstrap.cc:492] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: No bootstrap required, opened a new log
I20260809 04:56:49.642233 10234 ts_tablet_manager.cc:1397] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260809 04:56:49.646200 10235 tablet_bootstrap.cc:492] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: No bootstrap required, opened a new log
I20260809 04:56:49.646595 10235 ts_tablet_manager.cc:1397] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260809 04:56:49.648733 10238 tablet_bootstrap.cc:492] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: Bootstrap starting.
I20260809 04:56:49.653594 10238 tablet_bootstrap.cc:654] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:49.655450 10238 log.cc:824] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:49.657830 10234 raft_consensus.cc:348] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:49.658492 10234 raft_consensus.cc:374] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:49.658665 10234 raft_consensus.cc:729] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0490a08d31ba4a588b0a4129ab632520, State: Initialized, Role: FOLLOWER
I20260809 04:56:49.659345 10238 tablet_bootstrap.cc:492] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: No bootstrap required, opened a new log
I20260809 04:56:49.659711 10238 ts_tablet_manager.cc:1397] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.007s
I20260809 04:56:49.659520 10234 consensus_queue.cc:260] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:49.661501 10183 heartbeater.cc:503] Master 127.29.243.254:34625 was elected leader, sending a full tablet report...
I20260809 04:56:49.662221 10234 ts_tablet_manager.cc:1428] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: Time spent starting tablet: real 0.020s user 0.008s sys 0.012s
I20260809 04:56:49.662202 10235 raft_consensus.cc:348] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:49.662729 10235 raft_consensus.cc:374] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:49.662855 10235 raft_consensus.cc:729] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ff56221a0be4ba392f866e0ea37cfa6, State: Initialized, Role: FOLLOWER
I20260809 04:56:49.663659 10235 consensus_queue.cc:260] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:49.669615 10235 ts_tablet_manager.cc:1428] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: Time spent starting tablet: real 0.023s user 0.020s sys 0.004s
I20260809 04:56:49.669966 9961 heartbeater.cc:503] Master 127.29.243.254:34625 was elected leader, sending a full tablet report...
I20260809 04:56:49.679924 10238 raft_consensus.cc:348] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:49.680461 10238 raft_consensus.cc:374] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:49.680588 10238 raft_consensus.cc:729] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b24491f3874d1fa98090dc7e4ee226, State: Initialized, Role: FOLLOWER
I20260809 04:56:49.681300 10238 consensus_queue.cc:260] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:49.684037 10238 ts_tablet_manager.cc:1428] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: Time spent starting tablet: real 0.024s user 0.011s sys 0.008s
W20260809 04:56:49.684110 10185 tablet.cc:2369] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:56:49.823606 9964 tablet.cc:2369] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:56:49.896173 9713 tablet.cc:2369] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:56:49.934939 10241 raft_consensus.cc:482] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:49.935528 10241 raft_consensus.cc:504] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:49.937837 10241 leader_election.cc:288] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ff56221a0be4ba392f866e0ea37cfa6 (127.29.243.194:41757), 97b24491f3874d1fa98090dc7e4ee226 (127.29.243.193:45623)
I20260809 04:56:49.945473 10228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.938165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:53569 (local address 127.29.243.194:41757)
0809 04:56:49.938387 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:49.938392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:49.938417 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:56:49.938499 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:49.938504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:49.938612 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:49.938763 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:49.939319 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.940372 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.942815 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.944428 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.944589 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.944650 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:49.945115 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 04:56:49.945210 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:56:49.945647 10197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.937944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:41757 (local address 127.29.243.195:53569)
0809 04:56:49.938165 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:49.938184 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:49.938305 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:49.938766 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:49.938771 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:49.938797 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:49.939143 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.939158 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.940505 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.940510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.942627 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.942640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.944607 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.944615 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.944934 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.944973 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 04:56:49.945087 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 04:56:49.945227 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":2}
I20260809 04:56:49.946600 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.938877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:39969 (local address 127.29.243.193:45623)
0809 04:56:49.939189 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:49.939195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:49.939212 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:56:49.939307 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:49.939312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:49.939374 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:49.939449 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:49.940086 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.941322 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.943878 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.945436 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.945959 (+ 523us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.946045 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:49.946210 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 04:56:49.946335 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 04:56:49.946614 10189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:49.938334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45623 (local address 127.29.243.195:39969)
0809 04:56:49.938519 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:49.938535 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:49.938640 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:49.939485 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:49.939489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:49.939514 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:49.939916 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.939928 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.941482 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.941486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.943682 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:49.943698 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:49.945655 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:49.945661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:49.946064 (+ 403us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:49.946090 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:56:49.946167 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 04:56:49.946289 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 04:56:49.947219 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c6fd0b266a4fe885f1802a566aed06" candidate_uuid: "0490a08d31ba4a588b0a4129ab632520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" is_pre_election: true
I20260809 04:56:49.947957 9633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c6fd0b266a4fe885f1802a566aed06" candidate_uuid: "0490a08d31ba4a588b0a4129ab632520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b24491f3874d1fa98090dc7e4ee226" is_pre_election: true
I20260809 04:56:49.948024 9886 raft_consensus.cc:2454] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0490a08d31ba4a588b0a4129ab632520 in term 0.
I20260809 04:56:49.948689 9633 raft_consensus.cc:2454] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0490a08d31ba4a588b0a4129ab632520 in term 0.
I20260809 04:56:49.949052 10013 leader_election.cc:302] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0490a08d31ba4a588b0a4129ab632520, 7ff56221a0be4ba392f866e0ea37cfa6; no voters:
I20260809 04:56:49.949697 10241 raft_consensus.cc:2781] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:49.949828 10241 raft_consensus.cc:482] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:49.949991 10241 raft_consensus.cc:3037] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:49.953173 10241 raft_consensus.cc:504] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:49.954114 10241 leader_election.cc:288] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [CANDIDATE]: Term 1 election: Requested vote from peers 7ff56221a0be4ba392f866e0ea37cfa6 (127.29.243.194:41757), 97b24491f3874d1fa98090dc7e4ee226 (127.29.243.193:45623)
I20260809 04:56:49.954730 9633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c6fd0b266a4fe885f1802a566aed06" candidate_uuid: "0490a08d31ba4a588b0a4129ab632520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b24491f3874d1fa98090dc7e4ee226"
I20260809 04:56:49.954713 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c6fd0b266a4fe885f1802a566aed06" candidate_uuid: "0490a08d31ba4a588b0a4129ab632520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ff56221a0be4ba392f866e0ea37cfa6"
I20260809 04:56:49.954993 9633 raft_consensus.cc:3037] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:49.955066 9886 raft_consensus.cc:3037] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:49.958462 9633 raft_consensus.cc:2454] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0490a08d31ba4a588b0a4129ab632520 in term 1.
I20260809 04:56:49.958551 9886 raft_consensus.cc:2454] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0490a08d31ba4a588b0a4129ab632520 in term 1.
I20260809 04:56:49.958953 10013 leader_election.cc:302] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0490a08d31ba4a588b0a4129ab632520, 97b24491f3874d1fa98090dc7e4ee226; no voters:
I20260809 04:56:49.959311 10241 raft_consensus.cc:2781] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:49.959997 10241 raft_consensus.cc:686] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 1 LEADER]: Becoming Leader. State: Replica: 0490a08d31ba4a588b0a4129ab632520, State: Running, Role: LEADER
I20260809 04:56:49.960677 10241 consensus_queue.cc:237] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:49.967175 8162 catalog_manager.cc:5495] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0490a08d31ba4a588b0a4129ab632520 (127.29.243.195). New cstate: current_term: 1 leader_uuid: "0490a08d31ba4a588b0a4129ab632520" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } health_report { overall_health: HEALTHY } } }
I20260809 04:56:50.073918 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:50.068055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47982 (local address 127.29.243.195:37453)
0809 04:56:50.068285 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:50.068292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:50.068544 (+ 252us) server_negotiation.cc:407] Connection header received
0809 04:56:50.068762 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:50.068765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:50.068861 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:50.069002 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:50.069432 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:50.070140 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:50.071469 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:50.072002 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:50.072118 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:50.072625 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:50.073251 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:50.073392 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:50.073490 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 04:56:50.073572 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 04:56:50.083938 9633 raft_consensus.cc:1261] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226 [term 1 FOLLOWER]: Refusing update from remote peer 0490a08d31ba4a588b0a4129ab632520: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:50.084059 9886 raft_consensus.cc:1261] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 1 FOLLOWER]: Refusing update from remote peer 0490a08d31ba4a588b0a4129ab632520: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:50.085084 10246 consensus_queue.cc:1035] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:50.085649 10241 consensus_queue.cc:1035] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:50.101078 10251 mvcc.cc:205] Tried to move back new op lower bound from 7316485775692193792 to 7316485775203397632. Current Snapshot: MvccSnapshot[applied={T|T < 7316485775692193792}]
I20260809 04:56:50.120821 8162 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:50.121273 8162 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:50.130028 8162 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbf05d2d18743ec936aebcbe9ab5c80: Sending DeleteTablet for 3 replicas of tablet b3c6fd0b266a4fe885f1802a566aed06
I20260809 04:56:50.131379 9855 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c6fd0b266a4fe885f1802a566aed06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:50 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:58942
I20260809 04:56:50.131567 9598 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c6fd0b266a4fe885f1802a566aed06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:50 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:36202
I20260809 04:56:50.137187 10069 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c6fd0b266a4fe885f1802a566aed06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:50 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:47976
I20260809 04:56:50.138979 10268 tablet_replica.cc:330] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: stopping tablet replica
I20260809 04:56:50.139591 10268 raft_consensus.cc:2229] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:50.140539 10268 raft_consensus.cc:2258] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:50.143028 10268 ts_tablet_manager.cc:1905] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:50.149034 10269 tablet_replica.cc:330] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: stopping tablet replica
I20260809 04:56:50.152182 10268 ts_tablet_manager.cc:1918] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:50.152431 10268 log.cc:1196] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/b3c6fd0b266a4fe885f1802a566aed06
I20260809 04:56:50.153302 10268 ts_tablet_manager.cc:1939] T b3c6fd0b266a4fe885f1802a566aed06 P 7ff56221a0be4ba392f866e0ea37cfa6: Deleting consensus metadata
I20260809 04:56:50.154670 10269 raft_consensus.cc:2229] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:50.155591 8141 catalog_manager.cc:4841] TS 7ff56221a0be4ba392f866e0ea37cfa6 (127.29.243.194:41757): tablet b3c6fd0b266a4fe885f1802a566aed06 (table test-table [id=922aaa2e39cd4dfab8c21567bc535005]) successfully deleted
I20260809 04:56:50.157181 10269 raft_consensus.cc:2258] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:50.157624 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 8114
I20260809 04:56:50.158736 10270 tablet_replica.cc:330] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: stopping tablet replica
I20260809 04:56:50.159145 8203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:50.159314 10270 raft_consensus.cc:2229] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:50.160297 10270 raft_consensus.cc:2258] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:50.160512 10269 ts_tablet_manager.cc:1905] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:50.162218 10270 ts_tablet_manager.cc:1905] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:50.170702 10269 ts_tablet_manager.cc:1918] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:50.171061 10269 log.cc:1196] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/b3c6fd0b266a4fe885f1802a566aed06
I20260809 04:56:50.172346 10269 ts_tablet_manager.cc:1939] T b3c6fd0b266a4fe885f1802a566aed06 P 97b24491f3874d1fa98090dc7e4ee226: Deleting consensus metadata
I20260809 04:56:50.183624 10270 ts_tablet_manager.cc:1918] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:50.183975 10270 log.cc:1196] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/b3c6fd0b266a4fe885f1802a566aed06
I20260809 04:56:50.184963 10270 ts_tablet_manager.cc:1939] T b3c6fd0b266a4fe885f1802a566aed06 P 0490a08d31ba4a588b0a4129ab632520: Deleting consensus metadata
I20260809 04:56:50.188369 10272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:50.140804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60784 (local address 127.29.243.253:45079)
0809 04:56:50.157492 (+ 16688us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:50.157498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:50.157519 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:56:50.157583 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:50.157589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:50.157720 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:50.157833 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:50.161187 (+ 3354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:50.162712 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:50.166405 (+ 3693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:50.167287 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:50.167416 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:50.172765 (+ 5349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:50.172773 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:50.172779 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:50.172827 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:50.179075 (+ 6248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:50.180246 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:50.180252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:50.180256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:50.180527 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:50.181123 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:50.181127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:50.181131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:50.181605 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:50.187415 (+ 5810us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:50.187548 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 04:56:50.187831 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16556,"thread_start_us":34181,"threads_started":1}
I20260809 04:56:50.202557 10273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:50.156740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.29.243.252:39747)
0809 04:56:50.177046 (+ 20306us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:50.177052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:50.183330 (+ 6278us) server_negotiation.cc:407] Connection header received
0809 04:56:50.183409 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:50.183425 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:50.183560 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:50.183676 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:50.186478 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:50.187971 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:50.190511 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:50.191173 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:50.191333 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:50.195109 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:50.195115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:50.195118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:50.195152 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:50.199115 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:50.199993 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:50.199999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:50.200003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:50.200168 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:50.200624 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:50.200627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:50.200629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:50.201044 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:50.201416 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:50.201786 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 04:56:50.202041 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20078,"thread_start_us":19849,"threads_started":1}
I20260809 04:56:50.328058 8141 catalog_manager.cc:4841] TS 97b24491f3874d1fa98090dc7e4ee226 (127.29.243.193:45623): tablet b3c6fd0b266a4fe885f1802a566aed06 (table test-table [id=922aaa2e39cd4dfab8c21567bc535005]) successfully deleted
I20260809 04:56:50.329123 8142 catalog_manager.cc:4841] TS 0490a08d31ba4a588b0a4129ab632520 (127.29.243.195:37453): tablet b3c6fd0b266a4fe885f1802a566aed06 (table test-table [id=922aaa2e39cd4dfab8c21567bc535005]) successfully deleted
I20260809 04:56:50.329232 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 8114
I20260809 04:56:50.388765 10272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:50.364536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60798 (local address 127.29.243.253:45079)
0809 04:56:50.364766 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:50.364772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:50.371930 (+ 7158us) server_negotiation.cc:407] Connection header received
0809 04:56:50.372448 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:50.372455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:50.372545 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:50.372621 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:50.375095 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:50.376301 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:50.378983 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:50.379445 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:50.379560 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:50.382631 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:50.382636 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:50.382640 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:50.382674 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:50.385657 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:50.386578 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:50.386584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:50.386587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:50.386739 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:50.387195 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:50.387200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:50.387204 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:50.387576 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:50.387825 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:50.388192 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 04:56:50.388422 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:56:50.393986 10273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:50.364150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38782 (local address 127.29.243.252:39747)
0809 04:56:50.364362 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:50.364369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:50.378214 (+ 13845us) server_negotiation.cc:407] Connection header received
0809 04:56:50.378547 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:50.378553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:50.378672 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:50.378744 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:50.381255 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:50.382347 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:50.384443 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:50.385104 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:50.385234 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:50.387991 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:50.387996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:50.387999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:50.388030 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:50.391429 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:50.392071 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:50.392077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:50.392081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:50.392226 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:50.392570 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:50.392575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:50.392579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:50.392920 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:50.393182 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:50.393438 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 04:56:50.393662 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:56:51.106639 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.105756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.195:54425)
0809 04:56:51.106261 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.106393 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":269,"threads_started":1}
W20260809 04:56:51.107146 10183 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:34625 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
I20260809 04:56:51.109902 10285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.109123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.194:58941)
0809 04:56:51.109581 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.109697 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":195,"threads_started":1}
W20260809 04:56:51.110297 9961 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:34625 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
I20260809 04:56:51.115360 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.114648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.193:35091)
0809 04:56:51.115047 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.115156 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":194,"threads_started":1}
W20260809 04:56:51.115754 9711 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:34625 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
I20260809 04:56:51.629660 10288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbf05d2d18743ec936aebcbe9ab5c80)
I20260809 04:56:51.629959 10288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } }
I20260809 04:56:51.631492 10288 leader_election.cc:288] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbf05d2d18743ec936aebcbe9ab5c80 (127.29.243.254:34625), 5be2fa38862348589240908d3ba94459 (127.29.243.253:45079)
I20260809 04:56:51.632866 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.631604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52590)
0809 04:56:51.632486 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.632661 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":257,"threads_started":1}
W20260809 04:56:51.633184 9394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
I20260809 04:56:51.634043 10294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbf05d2d18743ec936aebcbe9ab5c80)
I20260809 04:56:51.634302 10294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } }
I20260809 04:56:51.635686 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.635173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52598)
0809 04:56:51.635423 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.635512 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260809 04:56:51.635913 9394 leader_election.cc:334] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1dbf05d2d18743ec936aebcbe9ab5c80 (127.29.243.254:34625): Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
I20260809 04:56:51.636708 10294 leader_election.cc:288] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbf05d2d18743ec936aebcbe9ab5c80 (127.29.243.254:34625), 3d05306c85104851ac7150793513ba54 (127.29.243.252:39747)
I20260809 04:56:51.637122 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.635501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52604)
0809 04:56:51.636536 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.636786 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":761,"threads_started":1}
W20260809 04:56:51.637398 9092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
I20260809 04:56:51.639207 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.638807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52616)
0809 04:56:51.639013 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.639085 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260809 04:56:51.639394 9092 leader_election.cc:334] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1dbf05d2d18743ec936aebcbe9ab5c80 (127.29.243.254:34625): Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
I20260809 04:56:51.640278 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.631793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:45079 (local address 127.0.0.1:60806)
0809 04:56:51.632540 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.632567 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:51.632719 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:51.633109 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:51.633114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:51.633143 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:51.633530 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:51.633545 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.635290 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.635297 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:51.637912 (+ 2615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:51.637929 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.639609 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.639616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:51.639823 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.639862 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 04:56:51.639951 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 04:56:51.640044 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":302,"threads_started":1}
I20260809 04:56:51.640465 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.631854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.29.243.253:45079)
0809 04:56:51.632687 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:51.632693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:51.632713 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:56:51.632863 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:51.632867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:51.632939 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:51.633876 (+ 937us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:51.634043 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.635138 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.638113 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.639379 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.639563 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.639635 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:51.639998 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 04:56:51.640132 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":491,"threads_started":1}
I20260809 04:56:51.641067 9131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d05306c85104851ac7150793513ba54" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5be2fa38862348589240908d3ba94459" is_pre_election: true
I20260809 04:56:51.641388 9131 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d05306c85104851ac7150793513ba54 in term 1.
I20260809 04:56:51.641892 9397 leader_election.cc:302] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d05306c85104851ac7150793513ba54, 5be2fa38862348589240908d3ba94459; no voters: 1dbf05d2d18743ec936aebcbe9ab5c80
I20260809 04:56:51.642272 10288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 04:56:51.642423 10288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dbf05d2d18743ec936aebcbe9ab5c80)
I20260809 04:56:51.642596 10288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:56:51.644608 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.635767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:39747 (local address 127.0.0.1:38792)
0809 04:56:51.637412 (+ 1645us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.637430 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:51.637547 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:51.638894 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:51.638899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:51.638936 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:51.639388 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:51.639398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.640899 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.640903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:51.642439 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:51.642455 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.644022 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.644026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:51.644157 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.644182 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:56:51.644258 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 04:56:51.644383 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1519,"thread_start_us":1300,"threads_started":1}
I20260809 04:56:51.644797 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.636238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38792 (local address 127.29.243.252:39747)
0809 04:56:51.638592 (+ 2354us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:51.638597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:51.638620 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:56:51.638676 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:51.638681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:51.638749 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:51.638863 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:51.639635 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.640768 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.642590 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.643851 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.644032 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.644109 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:51.644310 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 04:56:51.644439 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2247,"thread_start_us":2115,"threads_started":1}
I20260809 04:56:51.645439 9446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5be2fa38862348589240908d3ba94459" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3d05306c85104851ac7150793513ba54" is_pre_election: true
I20260809 04:56:51.645736 10288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } }
I20260809 04:56:51.646164 9446 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5be2fa38862348589240908d3ba94459 in current term 2: Already voted for candidate 3d05306c85104851ac7150793513ba54 in this term.
I20260809 04:56:51.646347 10288 leader_election.cc:288] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [CANDIDATE]: Term 2 election: Requested vote from peers 1dbf05d2d18743ec936aebcbe9ab5c80 (127.29.243.254:34625), 5be2fa38862348589240908d3ba94459 (127.29.243.253:45079)
I20260809 04:56:51.646834 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.646438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52622)
0809 04:56:51.646587 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.646674 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 04:56:51.646804 9095 leader_election.cc:302] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5be2fa38862348589240908d3ba94459; no voters: 1dbf05d2d18743ec936aebcbe9ab5c80, 3d05306c85104851ac7150793513ba54
I20260809 04:56:51.647094 10294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:56:51.647068 9131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d05306c85104851ac7150793513ba54" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5be2fa38862348589240908d3ba94459"
I20260809 04:56:51.648062 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.647664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52638)
0809 04:56:51.647810 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.647903 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 04:56:51.648232 9394 leader_election.cc:334] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1dbf05d2d18743ec936aebcbe9ab5c80 (127.29.243.254:34625): Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
I20260809 04:56:51.649590 10294 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:56:51.651705 9131 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d05306c85104851ac7150793513ba54 in term 2.
I20260809 04:56:51.652765 9397 leader_election.cc:302] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d05306c85104851ac7150793513ba54, 5be2fa38862348589240908d3ba94459; no voters: 1dbf05d2d18743ec936aebcbe9ab5c80
I20260809 04:56:51.653043 10288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:56:51.653453 10288 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [term 2 LEADER]: Becoming Leader. State: Replica: 3d05306c85104851ac7150793513ba54, State: Running, Role: LEADER
I20260809 04:56:51.653825 10288 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } }
I20260809 04:56:51.656203 10303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d05306c85104851ac7150793513ba54. Latest consensus state: current_term: 2 leader_uuid: "3d05306c85104851ac7150793513ba54" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:51.656503 10303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:51.657142 10307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:51.661409 10307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:51.662225 10307 catalog_manager.cc:1259] Loaded cluster ID: c12f80fb769e40f385feb51c1cf0ac40
I20260809 04:56:51.662297 10307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:51.663491 10307 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:51.664294 10307 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Loaded TSK: 0
I20260809 04:56:51.665197 10307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:56:51.676087 9425 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:51.684567 9425 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:51.688992 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.688600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52642)
0809 04:56:51.688771 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.688851 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 04:56:51.689038 9131 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [term 2 FOLLOWER]: Refusing update from remote peer 3d05306c85104851ac7150793513ba54: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 04:56:51.689843 10303 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 04:56:51.690018 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.689640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52644)
0809 04:56:51.689799 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.689884 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 04:56:51.690168 9394 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 -> Peer 1dbf05d2d18743ec936aebcbe9ab5c80 (127.29.243.254:34625): Couldn't send request to peer 1dbf05d2d18743ec936aebcbe9ab5c80. Status: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 04:56:51.692619 10294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d05306c85104851ac7150793513ba54. Latest consensus state: current_term: 2 leader_uuid: "3d05306c85104851ac7150793513ba54" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:51.693020 10294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:56:51.694301 10303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "3d05306c85104851ac7150793513ba54" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:51.694590 10303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:51.695144 10294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "3d05306c85104851ac7150793513ba54" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1dbf05d2d18743ec936aebcbe9ab5c80" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 34625 } } peers { permanent_uuid: "5be2fa38862348589240908d3ba94459" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 45079 } attrs { promote: false } } peers { permanent_uuid: "3d05306c85104851ac7150793513ba54" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 39747 } attrs { promote: false } } }
I20260809 04:56:51.695462 10294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be2fa38862348589240908d3ba94459 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:56:51.710526 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.703103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:41757 (local address 127.0.0.1:58952)
0809 04:56:51.703240 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.703256 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:51.703383 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:51.704272 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:51.704277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:51.704304 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:51.704561 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:51.704572 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.706124 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.706131 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:51.708145 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:51.708162 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.709709 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.709715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:51.709941 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.709983 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 04:56:51.710051 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 04:56:51.710190 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 04:56:51.712132 10320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.703480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58952 (local address 127.29.243.194:41757)
0809 04:56:51.703924 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:51.703929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:51.703948 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:56:51.704028 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:51.704033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:51.704105 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:51.704710 (+ 605us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:51.704885 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.705946 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.708366 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.709521 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.711418 (+ 1897us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.711562 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:51.711621 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 04:56:51.711775 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":233,"threads_started":1}
I20260809 04:56:51.713032 9855 tablet_service.cc:1467] Processing CreateTablet for tablet 7e313f4de2d74ec18669dcd7c3bea884 (DEFAULT_TABLE table=test-table [id=bf747a88c4f4455ca81f91719c4aa2ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:51.713553 9855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e313f4de2d74ec18669dcd7c3bea884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:51.721534 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.703606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45623 (local address 127.0.0.1:36216)
0809 04:56:51.703930 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.703951 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:51.704068 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:51.717121 (+ 13053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:51.717125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:51.717151 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:51.717422 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:51.717432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.718375 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.718380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:51.719715 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:51.719728 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.720871 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.720877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:51.721021 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.721046 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:56:51.721126 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:56:51.721264 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222}
I20260809 04:56:51.721576 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.704314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.29.243.193:45623)
0809 04:56:51.716838 (+ 12524us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:51.716843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:51.716860 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:56:51.716919 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:51.716923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:51.716981 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:51.717077 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:51.717563 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.718250 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.719869 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.720650 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.720797 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.720880 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:51.721175 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 04:56:51.721307 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12418,"thread_start_us":12867,"threads_started":1}
I20260809 04:56:51.722314 9598 tablet_service.cc:1467] Processing CreateTablet for tablet 7e313f4de2d74ec18669dcd7c3bea884 (DEFAULT_TABLE table=test-table [id=bf747a88c4f4455ca81f91719c4aa2ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:51.722759 9598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e313f4de2d74ec18669dcd7c3bea884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:51.723816 10326 tablet_bootstrap.cc:492] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6: Bootstrap starting.
I20260809 04:56:51.725600 10326 tablet_bootstrap.cc:654] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:51.727037 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.704032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:37453 (local address 127.0.0.1:47994)
0809 04:56:51.710765 (+ 6733us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.710784 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:51.710879 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:51.722144 (+ 11265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:51.722149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:51.722176 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:51.722345 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:51.722356 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.723681 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.723686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:51.725073 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:51.725086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.726222 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.726227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:51.726476 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.726509 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 04:56:51.726598 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 04:56:51.726721 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6627,"thread_start_us":12339,"threads_started":1}
I20260809 04:56:51.727967 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.704126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47994 (local address 127.29.243.195:37453)
0809 04:56:51.721706 (+ 17580us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:51.721712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:51.721732 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:56:51.721800 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:51.721806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:51.721876 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:51.722117 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:51.722494 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.723529 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.725240 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.726059 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.726203 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.726276 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:51.727517 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0809 04:56:51.727659 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17450,"thread_start_us":17843,"threads_started":1}
I20260809 04:56:51.728700 10326 tablet_bootstrap.cc:492] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6: No bootstrap required, opened a new log
I20260809 04:56:51.728773 10069 tablet_service.cc:1467] Processing CreateTablet for tablet 7e313f4de2d74ec18669dcd7c3bea884 (DEFAULT_TABLE table=test-table [id=bf747a88c4f4455ca81f91719c4aa2ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:51.728863 10326 ts_tablet_manager.cc:1397] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 04:56:51.729235 10069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e313f4de2d74ec18669dcd7c3bea884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:51.729732 10326 raft_consensus.cc:348] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:51.730008 10326 raft_consensus.cc:374] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:51.730105 10326 raft_consensus.cc:729] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ff56221a0be4ba392f866e0ea37cfa6, State: Initialized, Role: FOLLOWER
I20260809 04:56:51.730348 10326 consensus_queue.cc:260] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:51.731144 10326 ts_tablet_manager.cc:1428] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 04:56:51.732446 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.731413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.194:60949)
0809 04:56:51.731977 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.732155 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":281,"threads_started":1}
I20260809 04:56:51.734897 9960 heartbeater.cc:503] Master 127.29.243.252:39747 was elected leader, sending a full tablet report...
I20260809 04:56:51.735936 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.735511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.194:33603)
0809 04:56:51.735713 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.735792 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 04:56:51.736719 10331 tablet_bootstrap.cc:492] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520: Bootstrap starting.
I20260809 04:56:51.738497 10331 tablet_bootstrap.cc:654] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:51.739753 10332 tablet_bootstrap.cc:492] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226: Bootstrap starting.
I20260809 04:56:51.740724 10331 tablet_bootstrap.cc:492] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520: No bootstrap required, opened a new log
I20260809 04:56:51.740875 10331 ts_tablet_manager.cc:1397] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 04:56:51.741288 10332 tablet_bootstrap.cc:654] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:51.741621 10331 raft_consensus.cc:348] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:51.741842 10331 raft_consensus.cc:374] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:51.741943 10331 raft_consensus.cc:729] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0490a08d31ba4a588b0a4129ab632520, State: Initialized, Role: FOLLOWER
I20260809 04:56:51.742208 10331 consensus_queue.cc:260] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:51.743953 10331 ts_tablet_manager.cc:1428] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260809 04:56:51.744251 10332 tablet_bootstrap.cc:492] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226: No bootstrap required, opened a new log
I20260809 04:56:51.744460 10332 ts_tablet_manager.cc:1397] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:56:51.745297 10332 raft_consensus.cc:348] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:51.745621 10332 raft_consensus.cc:374] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:51.745770 10332 raft_consensus.cc:729] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b24491f3874d1fa98090dc7e4ee226, State: Initialized, Role: FOLLOWER
I20260809 04:56:51.746158 10182 heartbeater.cc:503] Master 127.29.243.252:39747 was elected leader, sending a full tablet report...
I20260809 04:56:51.746086 10332 consensus_queue.cc:260] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:51.748739 10332 ts_tablet_manager.cc:1428] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 04:56:51.750288 10336 raft_consensus.cc:482] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:51.750491 10336 raft_consensus.cc:504] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:51.751209 10336 leader_election.cc:288] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ff56221a0be4ba392f866e0ea37cfa6 (127.29.243.194:41757), 97b24491f3874d1fa98090dc7e4ee226 (127.29.243.193:45623)
I20260809 04:56:51.751773 9633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e313f4de2d74ec18669dcd7c3bea884" candidate_uuid: "0490a08d31ba4a588b0a4129ab632520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b24491f3874d1fa98090dc7e4ee226" is_pre_election: true
I20260809 04:56:51.752059 9633 raft_consensus.cc:2454] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0490a08d31ba4a588b0a4129ab632520 in term 0.
I20260809 04:56:51.752522 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e313f4de2d74ec18669dcd7c3bea884" candidate_uuid: "0490a08d31ba4a588b0a4129ab632520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" is_pre_election: true
I20260809 04:56:51.752806 9886 raft_consensus.cc:2454] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0490a08d31ba4a588b0a4129ab632520 in term 0.
I20260809 04:56:51.751785 9710 heartbeater.cc:503] Master 127.29.243.252:39747 was elected leader, sending a full tablet report...
I20260809 04:56:51.754180 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.744472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.195:59075)
0809 04:56:51.753706 (+ 9234us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.753897 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9107,"thread_start_us":8953,"threads_started":1}
I20260809 04:56:51.754231 10013 leader_election.cc:302] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0490a08d31ba4a588b0a4129ab632520, 97b24491f3874d1fa98090dc7e4ee226; no voters:
I20260809 04:56:51.754624 10336 raft_consensus.cc:2781] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:51.754738 10336 raft_consensus.cc:482] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:51.754827 10336 raft_consensus.cc:3037] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:51.756938 10336 raft_consensus.cc:504] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:51.757473 10336 leader_election.cc:288] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [CANDIDATE]: Term 1 election: Requested vote from peers 7ff56221a0be4ba392f866e0ea37cfa6 (127.29.243.194:41757), 97b24491f3874d1fa98090dc7e4ee226 (127.29.243.193:45623)
I20260809 04:56:51.758412 9633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e313f4de2d74ec18669dcd7c3bea884" candidate_uuid: "0490a08d31ba4a588b0a4129ab632520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b24491f3874d1fa98090dc7e4ee226"
I20260809 04:56:51.758610 9633 raft_consensus.cc:3037] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:51.758704 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.757959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.195:38073)
0809 04:56:51.758161 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.758247 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260809 04:56:51.759301 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e313f4de2d74ec18669dcd7c3bea884" candidate_uuid: "0490a08d31ba4a588b0a4129ab632520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ff56221a0be4ba392f866e0ea37cfa6"
I20260809 04:56:51.759514 9886 raft_consensus.cc:3037] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:51.761746 9633 raft_consensus.cc:2454] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0490a08d31ba4a588b0a4129ab632520 in term 1.
I20260809 04:56:51.762178 10013 leader_election.cc:302] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0490a08d31ba4a588b0a4129ab632520, 97b24491f3874d1fa98090dc7e4ee226; no voters:
I20260809 04:56:51.762414 9886 raft_consensus.cc:2454] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0490a08d31ba4a588b0a4129ab632520 in term 1.
I20260809 04:56:51.762825 10336 raft_consensus.cc:2781] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:51.763290 10336 raft_consensus.cc:686] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 1 LEADER]: Becoming Leader. State: Replica: 0490a08d31ba4a588b0a4129ab632520, State: Running, Role: LEADER
I20260809 04:56:51.763588 10336 consensus_queue.cc:237] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } }
I20260809 04:56:51.765185 10339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.749381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.193:35837)
0809 04:56:51.764786 (+ 15405us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.764912 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15289,"thread_start_us":15145,"threads_started":1}
I20260809 04:56:51.765339 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.763978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.195:44119)
0809 04:56:51.765035 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.765146 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":955}
I20260809 04:56:51.766681 10339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.766160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.193:39933)
0809 04:56:51.766293 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.766386 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 04:56:51.766492 9425 catalog_manager.cc:5495] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0490a08d31ba4a588b0a4129ab632520 (127.29.243.195). New cstate: current_term: 1 leader_uuid: "0490a08d31ba4a588b0a4129ab632520" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0490a08d31ba4a588b0a4129ab632520" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 37453 } health_report { overall_health: HEALTHY } } }
I20260809 04:56:51.794201 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.789809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48000 (local address 127.29.243.195:37453)
0809 04:56:51.789992 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:51.789996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:51.790013 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:56:51.790161 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:51.790164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:51.790232 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:51.790308 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:51.790611 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.791362 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.792661 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.793149 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.793244 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.793651 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:51.793756 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:51.793834 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:51.793928 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 04:56:51.793993 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:56:51.797427 9633 raft_consensus.cc:1261] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226 [term 1 FOLLOWER]: Refusing update from remote peer 0490a08d31ba4a588b0a4129ab632520: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:51.797488 9886 raft_consensus.cc:1261] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 1 FOLLOWER]: Refusing update from remote peer 0490a08d31ba4a588b0a4129ab632520: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:51.798058 10340 consensus_queue.cc:1035] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b24491f3874d1fa98090dc7e4ee226" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:51.798380 10336 consensus_queue.cc:1035] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ff56221a0be4ba392f866e0ea37cfa6" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 41757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:51.800750 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.800082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.194:42143)
0809 04:56:51.800413 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.800541 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209}
I20260809 04:56:51.801158 10339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.800695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.193:44765)
0809 04:56:51.800925 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.801025 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260809 04:56:51.807081 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.806321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.195:45575)
0809 04:56:51.806806 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.806931 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376}
I20260809 04:56:51.808542 10349 mvcc.cc:205] Tried to move back new op lower bound from 7316485782716194816 to 7316485782586822656. Current Snapshot: MvccSnapshot[applied={T|T < 7316485782716194816}]
I20260809 04:56:51.812029 10339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.811627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.193:36333)
0809 04:56:51.811772 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.811859 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 04:56:51.812855 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.812399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.195:48445)
0809 04:56:51.812607 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.812695 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 04:56:51.813428 9425 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:51.813769 9425 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:51.814344 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.813419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.29.243.194:45791)
0809 04:56:51.814104 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:51.814193 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":582}
I20260809 04:56:51.821398 9425 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54: Sending DeleteTablet for 3 replicas of tablet 7e313f4de2d74ec18669dcd7c3bea884
I20260809 04:56:51.821480 9598 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e313f4de2d74ec18669dcd7c3bea884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:51 UTC) from {username='oryx', principal='oryx/127.29.243.252@KRBTEST.COM'} at 127.0.0.1:36216
I20260809 04:56:51.821506 9855 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e313f4de2d74ec18669dcd7c3bea884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:51 UTC) from {username='oryx', principal='oryx/127.29.243.252@KRBTEST.COM'} at 127.0.0.1:58952
I20260809 04:56:51.822263 10069 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e313f4de2d74ec18669dcd7c3bea884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:51 UTC) from {username='oryx', principal='oryx/127.29.243.252@KRBTEST.COM'} at 127.0.0.1:47994
I20260809 04:56:51.822779 9854 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e313f4de2d74ec18669dcd7c3bea884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:51 UTC) from {username='oryx', principal='oryx/127.29.243.252@KRBTEST.COM'} at 127.0.0.1:58952
I20260809 04:56:51.822953 9596 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e313f4de2d74ec18669dcd7c3bea884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:51 UTC) from {username='oryx', principal='oryx/127.29.243.252@KRBTEST.COM'} at 127.0.0.1:36216
I20260809 04:56:51.827816 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 9370
I20260809 04:56:51.831059 10359 tablet_replica.cc:330] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226: stopping tablet replica
I20260809 04:56:51.831298 10359 raft_consensus.cc:2229] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:51.831643 10359 raft_consensus.cc:2258] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:51.837241 10361 tablet_replica.cc:330] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520: stopping tablet replica
I20260809 04:56:51.837441 10361 raft_consensus.cc:2229] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:51.837827 10361 raft_consensus.cc:2258] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:51.840039 10361 ts_tablet_manager.cc:1905] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:51.843717 10359 ts_tablet_manager.cc:1905] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:51.846179 10361 ts_tablet_manager.cc:1918] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:51.846333 10361 log.cc:1196] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/7e313f4de2d74ec18669dcd7c3bea884
I20260809 04:56:51.846793 10361 ts_tablet_manager.cc:1939] T 7e313f4de2d74ec18669dcd7c3bea884 P 0490a08d31ba4a588b0a4129ab632520: Deleting consensus metadata
I20260809 04:56:51.848132 9399 catalog_manager.cc:4841] TS 0490a08d31ba4a588b0a4129ab632520 (127.29.243.195:37453): tablet 7e313f4de2d74ec18669dcd7c3bea884 (table test-table [id=bf747a88c4f4455ca81f91719c4aa2ec]) successfully deleted
I20260809 04:56:51.851534 10359 ts_tablet_manager.cc:1918] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:51.851683 10359 log.cc:1196] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/7e313f4de2d74ec18669dcd7c3bea884
I20260809 04:56:51.851765 10360 tablet_replica.cc:330] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6: stopping tablet replica
I20260809 04:56:51.851982 10360 raft_consensus.cc:2229] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:51.852171 10359 ts_tablet_manager.cc:1939] T 7e313f4de2d74ec18669dcd7c3bea884 P 97b24491f3874d1fa98090dc7e4ee226: Deleting consensus metadata
I20260809 04:56:51.852285 10360 raft_consensus.cc:2258] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:51.853469 10360 ts_tablet_manager.cc:1905] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 04:56:51.853801 9397 catalog_manager.cc:4808] TS 97b24491f3874d1fa98090dc7e4ee226 (127.29.243.193:45623): delete failed for tablet 7e313f4de2d74ec18669dcd7c3bea884 because the tablet was not found. No further retry: Not found: Tablet not found: 7e313f4de2d74ec18669dcd7c3bea884
I20260809 04:56:51.856891 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.829054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36226 (local address 127.29.243.193:45623)
0809 04:56:51.829297 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:51.829303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:51.839924 (+ 10621us) server_negotiation.cc:407] Connection header received
0809 04:56:51.841032 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:51.841039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:51.841133 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:51.841221 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:51.843305 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.844372 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.846193 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.846834 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.846949 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.850074 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:51.850088 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:51.850107 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:51.850137 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:51.853959 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:51.854706 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:51.854716 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:51.854720 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:51.854887 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:51.855232 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:51.855237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:51.855240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:51.855719 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:51.856012 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:51.856329 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 04:56:51.856562 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:56:51.857565 9699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:51.858634 10360 ts_tablet_manager.cc:1918] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:51.858759 10360 log.cc:1196] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/7e313f4de2d74ec18669dcd7c3bea884
I20260809 04:56:51.859223 10360 ts_tablet_manager.cc:1939] T 7e313f4de2d74ec18669dcd7c3bea884 P 7ff56221a0be4ba392f866e0ea37cfa6: Deleting consensus metadata
W20260809 04:56:51.860673 9397 catalog_manager.cc:4808] TS 7ff56221a0be4ba392f866e0ea37cfa6 (127.29.243.194:41757): delete failed for tablet 7e313f4de2d74ec18669dcd7c3bea884 because the tablet was not found. No further retry: Not found: Tablet not found: 7e313f4de2d74ec18669dcd7c3bea884
I20260809 04:56:51.950294 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 9370
I20260809 04:56:51.977865 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 9736
I20260809 04:56:51.990464 10320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:51.978656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58964 (local address 127.29.243.194:41757)
0809 04:56:51.978837 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:51.978842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:51.978860 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:56:51.978995 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:51.979000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:51.979076 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:51.979161 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:51.980443 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.981141 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.982548 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:51.982972 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:51.983071 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:51.985512 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:51.985530 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:51.985549 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:51.985581 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:51.988241 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:51.988861 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:51.988869 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:51.988871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:51.988990 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:51.989259 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:51.989262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:51.989264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:51.989641 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:51.989865 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:51.990065 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 04:56:51.990225 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:56:51.991104 9950 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:52.086905 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 9736
I20260809 04:56:52.114750 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 9979
I20260809 04:56:52.129298 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:52.115471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48006 (local address 127.29.243.195:37453)
0809 04:56:52.115634 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:52.115639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:52.115708 (+ 69us) server_negotiation.cc:407] Connection header received
0809 04:56:52.115856 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:52.115860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:52.115954 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:52.116043 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:52.117815 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:52.118555 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:52.120242 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:52.120648 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:52.120747 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:52.123642 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:52.123658 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:52.123677 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:52.123710 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:52.126525 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:52.127237 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:52.127246 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:52.127248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:52.127504 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:52.127947 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:52.127952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:52.127954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:52.128330 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:52.128621 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:52.128848 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 04:56:52.129023 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:56:52.129979 10175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:52.248770 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 9979
I20260809 04:56:52.253062 10368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:52.249539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52650)
0809 04:56:52.252771 (+ 3232us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:52.252880 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3111,"thread_start_us":3008,"threads_started":1}
I20260809 04:56:52.254385 10368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:52.254017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:34625 (local address 127.0.0.1:52660)
0809 04:56:52.254160 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:52.254234 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:34625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 04:56:52.276059 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 8897
I20260809 04:56:52.277298 9168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:52.394845 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 8897
W20260809 04:56:52.424039 9397 connection.cc:535] client connection to 127.29.243.253:45079 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 04:56:52.424378 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 9066
I20260809 04:56:52.425380 10368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:52.424912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:45079 (local address 127.0.0.1:60814)
0809 04:56:52.425124 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:52.425206 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:45079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 04:56:52.425601 9397 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3d05306c85104851ac7150793513ba54 -> Peer 5be2fa38862348589240908d3ba94459 (127.29.243.253:45079): Couldn't send request to peer 5be2fa38862348589240908d3ba94459. Status: Network error: Client connection negotiation failed: client connection to 127.29.243.253:45079: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 04:56:52.425647 9475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:52.549108 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 9066
2026-08-09T04:56:52Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10272 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:56:52 dist-test-slave-21b9 krb5kdc[10379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:56:52 dist-test-slave-21b9 krb5kdc[10379](info): set up 2 sockets
Aug 09 04:56:52 dist-test-slave-21b9 krb5kdc[10379](info): commencing operation
krb5kdc: starting...
W20260809 04:56:53.783149 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:56:53 dist-test-slave-21b9 krb5kdc[10379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:56:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.254.keytab.
Entry for principal oryx/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.254.keytab.
I20260809 04:56:54.035110 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:37561
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.254.keytab
--principal=oryx/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:37561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:54.373123 10395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:54.373410 10395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:54.373543 10395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:54.380636 10395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:56:54.380739 10395 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:56:54.380788 10395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:54.380873 10395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:54.380935 10395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:56:54.381019 10395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:56:54.390726 10395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:37561
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.254.keytab
--principal=oryx/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:37561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:54.392016 10395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:54.393368 10395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:54.405165 10405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:54.405532 10408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:54.405884 10410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:54.406586 10395 server_base.cc:1034] running on GCE node
I20260809 04:56:54.407389 10395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:54.408830 10395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:54.410140 10395 hybrid_clock.cc:648] HybridClock initialized: now 1786251414410053 us; error 54 us; skew 500 ppm
I20260809 04:56:54.410478 10395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:54 dist-test-slave-21b9 krb5kdc[10379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251414, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:54.413724 10395 init.cc:377] Logged in from keytab as oryx/127.29.243.254@KRBTEST.COM (short username oryx)
I20260809 04:56:54.415232 10395 webserver.cc:460] Webserver started at http://127.29.243.254:34801/ using document root <none> and password file <none>
I20260809 04:56:54.415995 10395 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:54.416111 10395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:54.416443 10395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:54.419523 10395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "41b97b4693254b2b9541fc8144ab8607"
format_stamp: "Formatted at 2026-08-09 04:56:54 on dist-test-slave-21b9"
server_key: "e1e433f4452b213edc5994f93910c23e"
server_key_iv: "8919735fb5319544abe51469fb52cb1f"
server_key_version: "encryptionkey@0"
I20260809 04:56:54.420312 10395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "41b97b4693254b2b9541fc8144ab8607"
format_stamp: "Formatted at 2026-08-09 04:56:54 on dist-test-slave-21b9"
server_key: "e1e433f4452b213edc5994f93910c23e"
server_key_iv: "8919735fb5319544abe51469fb52cb1f"
server_key_version: "encryptionkey@0"
I20260809 04:56:54.425846 10395 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:56:54.429499 10423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:54.430933 10395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 04:56:54.431100 10395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "41b97b4693254b2b9541fc8144ab8607"
format_stamp: "Formatted at 2026-08-09 04:56:54 on dist-test-slave-21b9"
server_key: "e1e433f4452b213edc5994f93910c23e"
server_key_iv: "8919735fb5319544abe51469fb52cb1f"
server_key_version: "encryptionkey@0"
I20260809 04:56:54.431326 10395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:54.463981 10395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:54.469255 10395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:54.469515 10395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:54.489369 10395 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:37561
I20260809 04:56:54.489387 10515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:37561 every 8 connection(s)
I20260809 04:56:54.490749 10395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:56:54.491302 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 10395
I20260809 04:56:54.491598 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:56:54.492486 30671 external_mini_cluster.cc:1428] Setting key cbce19de6f010b14f673bed3133ae814
I20260809 04:56:54.495303 10516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:56:54 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:54.508090 10516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607: Bootstrap starting.
I20260809 04:56:54.512377 10516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:54.513583 10516 log.cc:824] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:54.517120 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:54.494799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55160 (local address 127.29.243.254:37561)
0809 04:56:54.495599 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:54.495611 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:54.495652 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:56:54.497253 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:54.497275 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:54.497667 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:54.497997 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:54.499785 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:54.501140 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:54.502894 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:54.503700 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:54.503843 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:54.508457 (+ 4614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:54.508469 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:54.508489 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:54.508534 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:54.512365 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:54.513071 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:54.513080 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:54.513088 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:54.513303 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:54.513675 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:54.513680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:54.513683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:54.514389 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:54.514620 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:54.515200 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 04:56:54.515771 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":318,"threads_started":1}
I20260809 04:56:54.517168 10516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607: No bootstrap required, opened a new log
I20260809 04:56:54.524652 10516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b97b4693254b2b9541fc8144ab8607" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37561 } }
I20260809 04:56:54.524976 10516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:54.525063 10516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41b97b4693254b2b9541fc8144ab8607, State: Initialized, Role: FOLLOWER
I20260809 04:56:54.525542 10516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b97b4693254b2b9541fc8144ab8607" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37561 } }
I20260809 04:56:54.525744 10516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:56:54.525840 10516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:56:54.525974 10516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:54.528086 10516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b97b4693254b2b9541fc8144ab8607" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37561 } }
I20260809 04:56:54.528539 10516 leader_election.cc:302] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41b97b4693254b2b9541fc8144ab8607; no voters:
I20260809 04:56:54.529209 10516 leader_election.cc:288] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:56:54.529345 10522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:54.530037 10522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [term 1 LEADER]: Becoming Leader. State: Replica: 41b97b4693254b2b9541fc8144ab8607, State: Running, Role: LEADER
I20260809 04:56:54.530601 10522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b97b4693254b2b9541fc8144ab8607" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37561 } }
I20260809 04:56:54.531479 10516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:56:54.533329 10525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41b97b4693254b2b9541fc8144ab8607. Latest consensus state: current_term: 1 leader_uuid: "41b97b4693254b2b9541fc8144ab8607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b97b4693254b2b9541fc8144ab8607" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37561 } } }
I20260809 04:56:54.533723 10525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:54.534102 10523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41b97b4693254b2b9541fc8144ab8607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b97b4693254b2b9541fc8144ab8607" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 37561 } } }
I20260809 04:56:54.534395 10523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607 [sys.catalog]: This master's current role is: LEADER
I20260809 04:56:54.536226 10530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:56:54.541798 10530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:56:54.550129 10530 catalog_manager.cc:1347] Generated new cluster ID: 40f2b0b96a1d43c5b89901086a56bbdb
I20260809 04:56:54.550239 10530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:56:54.569197 10530 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:56:54.570772 10530 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:56:54.582832 10530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607: Generated new TSK 0
I20260809 04:56:54.583784 10530 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab.
Entry for principal oryx/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab.
I20260809 04:56:54.655364 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:37561
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab
--principal=oryx/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:55.000800 10565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:55.001106 10565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:55.001250 10565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:55.008555 10565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:55.008677 10565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:55.008796 10565 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:56:55.008924 10565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:56:55.018748 10565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab
--principal=oryx/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.243.254:37561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:55.020128 10565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:55.021570 10565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:55.034475 10581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:55.034447 10586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:55.034420 10579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:55.035668 10565 server_base.cc:1034] running on GCE node
I20260809 04:56:55.036315 10565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:55.037292 10565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:55.038518 10565 hybrid_clock.cc:648] HybridClock initialized: now 1786251415038435 us; error 63 us; skew 500 ppm
I20260809 04:56:55.038812 10565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:55 dist-test-slave-21b9 krb5kdc[10379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251415, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:55.041953 10565 init.cc:377] Logged in from keytab as oryx/127.29.243.193@KRBTEST.COM (short username oryx)
I20260809 04:56:55.043350 10565 webserver.cc:460] Webserver started at http://127.29.243.193:44967/ using document root <none> and password file <none>
I20260809 04:56:55.044162 10565 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:55.044268 10565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:55.044601 10565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:55.047513 10565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "1b732d802a894d95a62575549c3af76d"
format_stamp: "Formatted at 2026-08-09 04:56:55 on dist-test-slave-21b9"
server_key: "5d07fafff3d0f55daba3cea50af5bc06"
server_key_iv: "ab59d90aec154885484bd82e1ebc1d43"
server_key_version: "encryptionkey@0"
I20260809 04:56:55.048293 10565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "1b732d802a894d95a62575549c3af76d"
format_stamp: "Formatted at 2026-08-09 04:56:55 on dist-test-slave-21b9"
server_key: "5d07fafff3d0f55daba3cea50af5bc06"
server_key_iv: "ab59d90aec154885484bd82e1ebc1d43"
server_key_version: "encryptionkey@0"
I20260809 04:56:55.053524 10565 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:56:55.057096 10596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:55.058413 10565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:56:55.058600 10565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "1b732d802a894d95a62575549c3af76d"
format_stamp: "Formatted at 2026-08-09 04:56:55 on dist-test-slave-21b9"
server_key: "5d07fafff3d0f55daba3cea50af5bc06"
server_key_iv: "ab59d90aec154885484bd82e1ebc1d43"
server_key_version: "encryptionkey@0"
I20260809 04:56:55.058811 10565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:55.088444 10565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:55.092710 10565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:55.093019 10565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:55.095794 10565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:55.095881 10565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:55.096009 10565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:55.096071 10565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:56:55 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251415, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:55.117362 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:55.096566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:55069 (local address 127.29.243.254:37561)
0809 04:56:55.097079 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:55.097085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:55.098218 (+ 1133us) server_negotiation.cc:407] Connection header received
0809 04:56:55.100296 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:55.100302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:55.100392 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:55.100498 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:55.102912 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.104077 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.105672 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.106372 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.106515 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:55.110668 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:55.110673 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:55.110677 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:55.110711 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:55.113774 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:55.114723 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:55.114729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:55.114732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:55.114893 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:55.115485 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:55.115491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:55.115495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:55.115886 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:55.116033 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:55.116777 (+ 744us) server_negotiation.cc:299] Negotiation successful
0809 04:56:55.117005 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":243,"threads_started":1}
I20260809 04:56:55.118642 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:55.097183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37561 (local address 127.29.243.193:55069)
0809 04:56:55.098026 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:55.098086 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:55.100021 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:55.100910 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:55.100922 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:55.101802 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:55.102648 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:55.102672 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.104231 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.104235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:55.105539 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:55.105552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.106598 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.106603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:55.106788 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:55.107563 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:55.107586 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:55.110438 (+ 2852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:55.113913 (+ 3475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:55.113923 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:55.113940 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:55.114585 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:55.115030 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:55.115036 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:55.115039 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:55.115346 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:55.116025 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:55.116033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:55.116395 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 04:56:55.116632 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 04:56:55.117171 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":347,"threads_started":1}
I20260809 04:56:55.139909 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:55.125485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:58129 (local address 127.29.243.254:37561)
0809 04:56:55.125655 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:55.125661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:55.126245 (+ 584us) server_negotiation.cc:407] Connection header received
0809 04:56:55.126402 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:55.126407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:55.126481 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:55.126560 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:55.127518 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.128364 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.130305 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.130821 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.130940 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:55.133623 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:55.133630 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:55.133635 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:55.133677 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:55.137234 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:55.137881 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:55.137888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:55.137892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:55.138066 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:55.138399 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:55.138404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:55.138407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:55.138774 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:55.138931 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:55.139289 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 04:56:55.139540 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:56:55.139952 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:55.125575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37561 (local address 127.29.243.193:58129)
0809 04:56:55.126146 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:55.126162 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:55.126296 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:55.126599 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:55.126603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:55.127052 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:55.127388 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:55.127399 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.128536 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.128541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:55.130114 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:55.130129 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.131219 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.131229 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:55.131443 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:55.132159 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:55.132178 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:55.133304 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:55.137360 (+ 4056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:55.137364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:55.137367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:55.137772 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:55.138180 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:55.138183 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:55.138186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:55.138311 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:55.138877 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:55.138882 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:55.139043 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 04:56:55.139152 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 04:56:55.139455 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":300,"threads_started":1}
I20260809 04:56:55.145706 10565 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:38973
I20260809 04:56:55.145725 10792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:38973 every 8 connection(s)
I20260809 04:56:55.146904 10565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:56:55.148981 10795 heartbeater.cc:347] Connected to a master server at 127.29.243.254:37561
I20260809 04:56:55.149209 10795 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:55.149852 10795 heartbeater.cc:511] Master 127.29.243.254:37561 requested a full tablet report, sending...
I20260809 04:56:55.151887 10448 ts_manager.cc:194] Registered new tserver with Master: 1b732d802a894d95a62575549c3af76d (127.29.243.193:38973)
I20260809 04:56:55.153302 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 10565
I20260809 04:56:55.153354 10448 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:55069
I20260809 04:56:55.153532 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:56:55.154129 30671 external_mini_cluster.cc:1428] Setting key 772dd0d5d9fadf778189e48f20df962c
WARNING: no policy specified for oryx/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab.
Entry for principal oryx/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab.
I20260809 04:56:55.208551 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:37561
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab
--principal=oryx/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:55.553737 10800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:55.554052 10800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:55.554215 10800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:55.561784 10800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:55.561931 10800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:55.562063 10800 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:56:55.562160 10800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:56:55.572263 10800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab
--principal=oryx/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.243.254:37561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:55.573598 10800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:55.575006 10800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:55.588318 10808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:55.588294 10811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:55.588568 10809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:55.589104 10800 server_base.cc:1034] running on GCE node
I20260809 04:56:55.589808 10800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:55.591300 10800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:55.592540 10800 hybrid_clock.cc:648] HybridClock initialized: now 1786251415592426 us; error 106 us; skew 500 ppm
I20260809 04:56:55.592835 10800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:55 dist-test-slave-21b9 krb5kdc[10379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251415, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:55.596359 10800 init.cc:377] Logged in from keytab as oryx/127.29.243.194@KRBTEST.COM (short username oryx)
I20260809 04:56:55.600028 10800 webserver.cc:460] Webserver started at http://127.29.243.194:42381/ using document root <none> and password file <none>
I20260809 04:56:55.600886 10800 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:55.600994 10800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:55.601286 10800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:55.604426 10800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "ddde5dc74c934a508fe7228a1d83a666"
format_stamp: "Formatted at 2026-08-09 04:56:55 on dist-test-slave-21b9"
server_key: "6aecbade992f606a91dc2b28e9640bf4"
server_key_iv: "c191718e09f2c512873911a714d83c1c"
server_key_version: "encryptionkey@0"
I20260809 04:56:55.605171 10800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "ddde5dc74c934a508fe7228a1d83a666"
format_stamp: "Formatted at 2026-08-09 04:56:55 on dist-test-slave-21b9"
server_key: "6aecbade992f606a91dc2b28e9640bf4"
server_key_iv: "c191718e09f2c512873911a714d83c1c"
server_key_version: "encryptionkey@0"
I20260809 04:56:55.610271 10800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:56:55.613834 10821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:55.615149 10800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:56:55.615341 10800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "ddde5dc74c934a508fe7228a1d83a666"
format_stamp: "Formatted at 2026-08-09 04:56:55 on dist-test-slave-21b9"
server_key: "6aecbade992f606a91dc2b28e9640bf4"
server_key_iv: "c191718e09f2c512873911a714d83c1c"
server_key_version: "encryptionkey@0"
I20260809 04:56:55.615559 10800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:55.640579 10800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:55.644896 10800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:55.645169 10800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:55.647878 10800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:55.647972 10800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:55.648099 10800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:55.648156 10800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:56:55 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251415, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:55.671994 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:55.648524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:58669 (local address 127.29.243.254:37561)
0809 04:56:55.648987 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:55.648993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:55.650236 (+ 1243us) server_negotiation.cc:407] Connection header received
0809 04:56:55.652416 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:55.652421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:55.652512 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:55.652608 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:55.654713 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.655951 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.658941 (+ 2990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.659616 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.659775 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:55.665043 (+ 5268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:55.665051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:55.665056 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:55.665103 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:55.668328 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:55.669317 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:55.669323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:55.669326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:55.669518 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:55.669980 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:55.669985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:55.669989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:55.670373 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:55.670548 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:55.671230 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 04:56:55.671582 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":215,"threads_started":1}
I20260809 04:56:55.673331 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:55.649209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37561 (local address 127.29.243.194:58669)
0809 04:56:55.650043 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:55.650093 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:55.652135 (+ 2042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:55.652911 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:55.652926 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:55.653685 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:55.654493 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:55.654524 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.656178 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.656184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:55.658697 (+ 2513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:55.658712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.660143 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.660152 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:55.660501 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:55.661392 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:55.661430 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:55.664788 (+ 3358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:55.668476 (+ 3688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:55.668487 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:55.668508 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:55.669156 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:55.669662 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:55.669668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:55.669672 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:55.669879 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:55.670510 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:55.670516 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:55.670863 (+ 347us) client_negotiation.cc:769] Sending connection context
0809 04:56:55.671101 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 04:56:55.671669 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":343,"threads_started":1}
I20260809 04:56:55.688793 10800 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:45111
I20260809 04:56:55.688925 11021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:45111 every 8 connection(s)
I20260809 04:56:55.690150 10800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:56:55.692389 11023 heartbeater.cc:347] Connected to a master server at 127.29.243.254:37561
I20260809 04:56:55.692642 11023 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:55.693499 11023 heartbeater.cc:511] Master 127.29.243.254:37561 requested a full tablet report, sending...
I20260809 04:56:55.694128 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 10800
I20260809 04:56:55.694335 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:56:55.694864 30671 external_mini_cluster.cc:1428] Setting key 40c690f4b3054a40bbf60102c34e21de
I20260809 04:56:55.695554 10448 ts_manager.cc:194] Registered new tserver with Master: ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111)
I20260809 04:56:55.696695 10448 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:58669
I20260809 04:56:55.702000 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:55.681553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:56311 (local address 127.29.243.254:37561)
0809 04:56:55.681758 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:55.681764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:55.687963 (+ 6199us) server_negotiation.cc:407] Connection header received
0809 04:56:55.688252 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:55.688257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:55.688324 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:55.688404 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:55.689536 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.690231 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.692174 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.692826 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.692953 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:55.696766 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:55.696772 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:55.696776 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:55.696815 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:55.699492 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:55.700122 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:55.700126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:55.700128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:55.700246 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:55.700599 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:55.700603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:55.700605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:55.700912 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:55.701043 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:55.701461 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 04:56:55.701684 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:56:55.702065 10981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:55.681633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37561 (local address 127.29.243.194:56311)
0809 04:56:55.687835 (+ 6202us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:55.687856 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:55.688106 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:55.688456 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:55.688466 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:55.689030 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:55.689371 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:55.689382 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.690354 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.690359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:55.691972 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:55.691987 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:55.693214 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:55.693222 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:55.693408 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:55.695446 (+ 2038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:55.695489 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:55.696548 (+ 1059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:55.699618 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:55.699624 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:55.699629 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:55.700003 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:55.700349 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:55.700354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:55.700358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:55.700491 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:55.701035 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:55.701039 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:55.701204 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 04:56:55.701315 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 04:56:55.701613 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6029,"thread_start_us":5830,"threads_started":1}
WARNING: no policy specified for oryx/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab.
Entry for principal oryx/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab.
I20260809 04:56:55.751427 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:37561
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab
--principal=oryx/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:56:56.097069 11027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:56:56.097364 11027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:56:56.097512 11027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:56:56.104687 11027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:56:56.104802 11027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:56:56.104933 11027 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:56:56.105044 11027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:56:56.114471 11027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab
--principal=oryx/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.243.254:37561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:56:56.115859 11027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:56:56.117209 11027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:56:56.130108 11034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:56.130129 11039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:56:56.130067 11035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:56:56.130846 11027 server_base.cc:1034] running on GCE node
I20260809 04:56:56.131476 11027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:56:56.132404 11027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:56:56.133620 11027 hybrid_clock.cc:648] HybridClock initialized: now 1786251416133547 us; error 47 us; skew 500 ppm
I20260809 04:56:56.133898 11027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:56:56 dist-test-slave-21b9 krb5kdc[10379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251416, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:56:56.136982 11027 init.cc:377] Logged in from keytab as oryx/127.29.243.195@KRBTEST.COM (short username oryx)
I20260809 04:56:56.138396 11027 webserver.cc:460] Webserver started at http://127.29.243.195:44121/ using document root <none> and password file <none>
I20260809 04:56:56.139072 11027 fs_manager.cc:359] Metadata directory not provided
I20260809 04:56:56.139173 11027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:56:56.139518 11027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:56:56.142367 11027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "c50f84ec8edc4440a83dc942c1bbd55f"
format_stamp: "Formatted at 2026-08-09 04:56:56 on dist-test-slave-21b9"
server_key: "036a2b947fb2d28e9552266ca82a43cb"
server_key_iv: "14d6f135c4570d33f3516d97e667011f"
server_key_version: "encryptionkey@0"
I20260809 04:56:56.143129 11027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "c50f84ec8edc4440a83dc942c1bbd55f"
format_stamp: "Formatted at 2026-08-09 04:56:56 on dist-test-slave-21b9"
server_key: "036a2b947fb2d28e9552266ca82a43cb"
server_key_iv: "14d6f135c4570d33f3516d97e667011f"
server_key_version: "encryptionkey@0"
I20260809 04:56:56.148288 11027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 04:56:56.151669 11048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:56.152933 11027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 04:56:56.153087 11027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "c50f84ec8edc4440a83dc942c1bbd55f"
format_stamp: "Formatted at 2026-08-09 04:56:56 on dist-test-slave-21b9"
server_key: "036a2b947fb2d28e9552266ca82a43cb"
server_key_iv: "14d6f135c4570d33f3516d97e667011f"
server_key_version: "encryptionkey@0"
I20260809 04:56:56.153285 11027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:56:56.156273 10795 heartbeater.cc:503] Master 127.29.243.254:37561 was elected leader, sending a full tablet report...
I20260809 04:56:56.173022 11027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:56:56.177192 11027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:56:56.177450 11027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:56:56.180212 11027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:56:56.180321 11027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:56:56.180459 11027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:56:56.180540 11027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:56:56 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251416, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:56.206053 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.180641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:40087 (local address 127.29.243.254:37561)
0809 04:56:56.180840 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:56.180845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:56.182459 (+ 1614us) server_negotiation.cc:407] Connection header received
0809 04:56:56.184681 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:56.184687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:56.184799 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:56.184937 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:56.187330 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.188329 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.191257 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.191942 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.192079 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.197107 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:56.197114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:56.197119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:56.197167 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:56.200722 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:56.201782 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:56.201789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:56.201793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:56.201980 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:56.202516 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:56.202522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:56.202525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:56.202939 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:56.203097 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:56.205497 (+ 2400us) server_negotiation.cc:299] Negotiation successful
0809 04:56:56.205758 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:56:56.206409 11062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.181377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37561 (local address 127.29.243.195:40087)
0809 04:56:56.182266 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:56.182307 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:56.184365 (+ 2058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:56.185245 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:56.185258 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:56.186092 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:56.187063 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.187099 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.188530 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.188539 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.191039 (+ 2500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.191058 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.192261 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.192274 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.192558 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.193441 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:56.193479 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:56.196836 (+ 3357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:56.200877 (+ 4041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:56.200891 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:56.200913 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:56.201621 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:56.202129 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:56.202135 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:56.202140 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:56.202379 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:56.203100 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:56.203109 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:56.203574 (+ 465us) client_negotiation.cc:769] Sending connection context
0809 04:56:56.203961 (+ 387us) client_negotiation.cc:240] Negotiation successful
0809 04:56:56.204550 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":376,"threads_started":1}
I20260809 04:56:56.222788 11027 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:41889
I20260809 04:56:56.222813 11230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:41889 every 8 connection(s)
I20260809 04:56:56.224427 11027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:56:56.226994 11231 heartbeater.cc:347] Connected to a master server at 127.29.243.254:37561
I20260809 04:56:56.227227 11231 heartbeater.cc:464] Registering TS with master...
I20260809 04:56:56.228003 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 11027
I20260809 04:56:56.228087 11231 heartbeater.cc:511] Master 127.29.243.254:37561 requested a full tablet report, sending...
I20260809 04:56:56.228197 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:56:56.228623 30671 external_mini_cluster.cc:1428] Setting key 294001be5598f8a4bf780c46820069e1
I20260809 04:56:56.229468 11197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.213723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37561 (local address 127.29.243.195:36451)
0809 04:56:56.214444 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:56.214466 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:56.214608 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:56.214943 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:56.214948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:56.215473 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:56.216057 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.216077 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.217618 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.217625 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.219530 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.219546 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.220806 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.220813 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.220966 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.221611 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:56.221637 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:56.222825 (+ 1188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:56.226540 (+ 3715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:56.226557 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:56.226561 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:56.227070 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:56.227512 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:56.227516 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:56.227518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:56.227692 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:56.228398 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:56.228403 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:56.228602 (+ 199us) client_negotiation.cc:769] Sending connection context
0809 04:56:56.228744 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 04:56:56.229053 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":385,"threads_started":1}
I20260809 04:56:56.229768 10448 ts_manager.cc:194] Registered new tserver with Master: c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195:41889)
I20260809 04:56:56.230278 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.213546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:36451 (local address 127.29.243.254:37561)
0809 04:56:56.213734 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:56.213740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:56.214554 (+ 814us) server_negotiation.cc:407] Connection header received
0809 04:56:56.214724 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:56.214730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:56.214805 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:56.214902 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:56.216218 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.217317 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.219733 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.220424 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.220566 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.223061 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:56.223068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:56.223073 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:56.223111 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:56.226348 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:56.227173 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:56.227178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:56.227182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:56.227375 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:56.227821 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:56.227826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:56.227829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:56.228187 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:56.228334 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:56.229753 (+ 1419us) server_negotiation.cc:299] Negotiation successful
0809 04:56:56.229953 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 04:56:56.230909 10448 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:40087
I20260809 04:56:56.232086 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:56:56 dist-test-slave-21b9 krb5kdc[10379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251416, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:56:56 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:56:56 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:56:56.290541 10837 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:56:56.279405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.29.243.254:37561)
0809 04:56:56.279642 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:56.279655 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:56.279838 (+ 183us) server_negotiation.cc:407] Connection header received
0809 04:56:56.280044 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:56.280049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:56.280121 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:56.280214 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:56.281744 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.282537 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.283947 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.284372 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.284504 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.290203 (+ 5699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55166: BlockingRecv error: recv got EOF from 127.0.0.1:55166 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 04:56:56 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251416, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:56.306267 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.293535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55172 (local address 127.29.243.254:37561)
0809 04:56:56.293741 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:56.293747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:56.293934 (+ 187us) server_negotiation.cc:407] Connection header received
0809 04:56:56.294077 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:56.294081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:56.294142 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:56.294220 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:56.295386 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.296110 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.297326 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.297777 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.297864 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.301474 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:56.301479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:56.301482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:56.301515 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:56.304002 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:56.304674 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:56.304677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:56.304680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:56.304796 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:56.305096 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:56.305099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:56.305100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:56.305350 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:56.305558 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:56.305820 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 04:56:56.305983 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:56:56.310519 10448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:56.317411 10448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:56.353945 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.330355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55484 (local address 127.29.243.193:38973)
0809 04:56:56.342683 (+ 12328us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:56.342695 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:56.343633 (+ 938us) server_negotiation.cc:407] Connection header received
0809 04:56:56.344706 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:56.344715 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:56.344972 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:56.345165 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:56.346221 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.348036 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.350839 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.352328 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.352472 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.352658 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:56.353430 (+ 772us) server_negotiation.cc:299] Negotiation successful
0809 04:56:56.353605 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12125,"thread_start_us":12340,"threads_started":1}
I20260809 04:56:56.354352 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.331459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49686 (local address 127.29.243.195:41889)
0809 04:56:56.344499 (+ 13040us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:56.344507 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:56.344540 (+ 33us) server_negotiation.cc:407] Connection header received
0809 04:56:56.344645 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:56.344653 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:56.344938 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:56.345249 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:56.346353 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.348231 (+ 1878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.350857 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.352328 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.352503 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.352658 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:56.353695 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0809 04:56:56.353859 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12875,"thread_start_us":12705,"threads_started":1}
I20260809 04:56:56.354450 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.330871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41889 (local address 127.0.0.1:49686)
0809 04:56:56.344082 (+ 13211us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:56.344104 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:56.344273 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:56.345571 (+ 1298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:56.345576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:56.345606 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:56.346108 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.346121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.348411 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.348418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.350694 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.350707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.352821 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.352831 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.353134 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.353214 (+ 80us) client_negotiation.cc:769] Sending connection context
0809 04:56:56.353368 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 04:56:56.353571 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13074,"mutex_wait_us":141,"thread_start_us":12935,"threads_started":1}
I20260809 04:56:56.354010 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.330176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38973 (local address 127.0.0.1:55484)
0809 04:56:56.343474 (+ 13298us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:56.343526 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:56.344496 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:56.345451 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:56.345459 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:56.345492 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:56.345973 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.345997 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.348237 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.348243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.350681 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.350693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.352853 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.352859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.353103 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.353214 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 04:56:56.353368 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 04:56:56.353571 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13033,"mutex_wait_us":136,"thread_start_us":12887,"threads_started":1}
I20260809 04:56:56.356297 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.344739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:45111 (local address 127.0.0.1:36826)
0809 04:56:56.349405 (+ 4666us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:56.349427 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:56.349557 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:56.350158 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:56.350162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:56.350191 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:56.350637 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.350650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.352086 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.352090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.353757 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.353773 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.355533 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.355540 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.355701 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.355732 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 04:56:56.355812 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:56:56.355942 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4540,"mutex_wait_us":1,"thread_start_us":4395,"threads_started":1}
I20260809 04:56:56.356345 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.344188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36826 (local address 127.29.243.194:45111)
0809 04:56:56.349058 (+ 4870us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:56.349091 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:56.349537 (+ 446us) server_negotiation.cc:407] Connection header received
0809 04:56:56.349719 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:56.349727 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:56.349966 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:56.350149 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:56.350814 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.351928 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.353953 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.355328 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.355493 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.355662 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:56.355880 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 04:56:56.356040 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4704,"thread_start_us":4568,"threads_started":1}
I20260809 04:56:56.358587 11126 tablet_service.cc:1467] Processing CreateTablet for tablet a568fc239a39445e9a6b5bfec9ddf080 (DEFAULT_TABLE table=test-table [id=bc01b8162162405794a34203a1359428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:56.358587 10674 tablet_service.cc:1467] Processing CreateTablet for tablet a568fc239a39445e9a6b5bfec9ddf080 (DEFAULT_TABLE table=test-table [id=bc01b8162162405794a34203a1359428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:56.360121 10674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a568fc239a39445e9a6b5bfec9ddf080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:56.360133 11126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a568fc239a39445e9a6b5bfec9ddf080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:56.359958 10904 tablet_service.cc:1467] Processing CreateTablet for tablet a568fc239a39445e9a6b5bfec9ddf080 (DEFAULT_TABLE table=test-table [id=bc01b8162162405794a34203a1359428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:56.361059 10904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a568fc239a39445e9a6b5bfec9ddf080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:56.371745 11266 tablet_bootstrap.cc:492] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: Bootstrap starting.
I20260809 04:56:56.374934 11266 tablet_bootstrap.cc:654] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:56.376281 11266 log.cc:824] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:56.376708 11267 tablet_bootstrap.cc:492] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: Bootstrap starting.
I20260809 04:56:56.377087 11268 tablet_bootstrap.cc:492] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: Bootstrap starting.
I20260809 04:56:56.380167 11266 tablet_bootstrap.cc:492] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: No bootstrap required, opened a new log
I20260809 04:56:56.380461 11266 ts_tablet_manager.cc:1397] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260809 04:56:56.380600 11268 tablet_bootstrap.cc:654] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:56.381479 11267 tablet_bootstrap.cc:654] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:56.381876 11268 log.cc:824] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:56.382447 11267 log.cc:824] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:56:56.384888 11268 tablet_bootstrap.cc:492] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: No bootstrap required, opened a new log
I20260809 04:56:56.385177 11268 ts_tablet_manager.cc:1397] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 04:56:56.385390 11267 tablet_bootstrap.cc:492] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: No bootstrap required, opened a new log
I20260809 04:56:56.385651 11267 ts_tablet_manager.cc:1397] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260809 04:56:56.388585 11266 raft_consensus.cc:348] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.388950 11266 raft_consensus.cc:374] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:56.389045 11266 raft_consensus.cc:729] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddde5dc74c934a508fe7228a1d83a666, State: Initialized, Role: FOLLOWER
I20260809 04:56:56.389650 11266 consensus_queue.cc:260] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.390720 11023 heartbeater.cc:503] Master 127.29.243.254:37561 was elected leader, sending a full tablet report...
I20260809 04:56:56.392300 11266 ts_tablet_manager.cc:1428] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260809 04:56:56.395846 11268 raft_consensus.cc:348] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.396303 11268 raft_consensus.cc:374] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:56.396409 11268 raft_consensus.cc:729] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b732d802a894d95a62575549c3af76d, State: Initialized, Role: FOLLOWER
I20260809 04:56:56.396510 11267 raft_consensus.cc:348] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.396906 11267 raft_consensus.cc:374] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:56.397004 11267 raft_consensus.cc:729] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c50f84ec8edc4440a83dc942c1bbd55f, State: Initialized, Role: FOLLOWER
I20260809 04:56:56.397050 11268 consensus_queue.cc:260] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.397585 11267 consensus_queue.cc:260] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.398662 11231 heartbeater.cc:503] Master 127.29.243.254:37561 was elected leader, sending a full tablet report...
I20260809 04:56:56.398808 11268 ts_tablet_manager.cc:1428] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260809 04:56:56.399252 11267 ts_tablet_manager.cc:1428] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: Time spent starting tablet: real 0.013s user 0.006s sys 0.008s
W20260809 04:56:56.399252 10796 tablet.cc:2369] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:56:56.442276 11024 tablet.cc:2369] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:56:56.476760 11232 tablet.cc:2369] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:56:56.587587 11279 raft_consensus.cc:482] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:56.587926 11279 raft_consensus.cc:504] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.589864 11279 leader_election.cc:288] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111), c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195:41889)
I20260809 04:56:56.596329 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.590307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:39875 (local address 127.29.243.194:45111)
0809 04:56:56.590763 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:56.590769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:56.591157 (+ 388us) server_negotiation.cc:407] Connection header received
0809 04:56:56.591407 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:56.591411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:56.591475 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:56.591549 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:56.592146 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.592850 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.594662 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.595451 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.595606 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.595681 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:56.595959 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 04:56:56.596077 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301}
I20260809 04:56:56.596405 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.590142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:45111 (local address 127.29.243.193:39875)
0809 04:56:56.591016 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:56.591043 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:56.591223 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:56.591580 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:56.591585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:56.591618 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:56.592004 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.592015 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.592976 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.592981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.594510 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.594522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.595660 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.595666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.595812 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.595839 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:56:56.595922 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 04:56:56.596048 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":539,"threads_started":1}
I20260809 04:56:56.596414 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.590450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:53267 (local address 127.29.243.195:41889)
0809 04:56:56.590635 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:56.590641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:56.591421 (+ 780us) server_negotiation.cc:407] Connection header received
0809 04:56:56.591576 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:56.591580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:56.591640 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:56.591711 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:56.592380 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.593056 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.594730 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.595526 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.595678 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.595747 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:56.596040 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 04:56:56.596159 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 04:56:56.596632 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.590378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41889 (local address 127.29.243.193:53267)
0809 04:56:56.591315 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:56.591332 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:56.591457 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:56.591750 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:56.591755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:56.591780 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:56.592194 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.592204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.593193 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.593199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.594577 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.594590 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.595740 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.595746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.595898 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.595921 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:56:56.596000 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:56:56.596117 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":568,"threads_started":1}
I20260809 04:56:56.597498 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a568fc239a39445e9a6b5bfec9ddf080" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddde5dc74c934a508fe7228a1d83a666" is_pre_election: true
I20260809 04:56:56.597568 11159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a568fc239a39445e9a6b5bfec9ddf080" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" is_pre_election: true
I20260809 04:56:56.597970 10939 raft_consensus.cc:2454] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 0.
I20260809 04:56:56.598027 11159 raft_consensus.cc:2454] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 0.
I20260809 04:56:56.598632 10599 leader_election.cc:302] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b732d802a894d95a62575549c3af76d, c50f84ec8edc4440a83dc942c1bbd55f; no voters:
I20260809 04:56:56.599076 11279 raft_consensus.cc:2781] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:56.599231 11279 raft_consensus.cc:482] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:56.599372 11279 raft_consensus.cc:3037] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:56.601912 11279 raft_consensus.cc:504] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.602571 11279 leader_election.cc:288] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 election: Requested vote from peers ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111), c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195:41889)
I20260809 04:56:56.603008 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a568fc239a39445e9a6b5bfec9ddf080" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddde5dc74c934a508fe7228a1d83a666"
I20260809 04:56:56.603227 10939 raft_consensus.cc:3037] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:56.603744 11159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a568fc239a39445e9a6b5bfec9ddf080" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c50f84ec8edc4440a83dc942c1bbd55f"
I20260809 04:56:56.604022 11159 raft_consensus.cc:3037] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:56.606555 10939 raft_consensus.cc:2454] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 1.
I20260809 04:56:56.607050 11159 raft_consensus.cc:2454] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 1.
I20260809 04:56:56.607065 10600 leader_election.cc:302] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b732d802a894d95a62575549c3af76d, ddde5dc74c934a508fe7228a1d83a666; no voters:
I20260809 04:56:56.607826 11279 raft_consensus.cc:2781] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:56.608290 11279 raft_consensus.cc:686] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 1 LEADER]: Becoming Leader. State: Replica: 1b732d802a894d95a62575549c3af76d, State: Running, Role: LEADER
I20260809 04:56:56.609058 11279 consensus_queue.cc:237] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.613966 10448 catalog_manager.cc:5495] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b732d802a894d95a62575549c3af76d (127.29.243.193). New cstate: current_term: 1 leader_uuid: "1b732d802a894d95a62575549c3af76d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } health_report { overall_health: UNKNOWN } } }
I20260809 04:56:56.633471 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.628434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38804 (local address 127.29.243.193:38973)
0809 04:56:56.628612 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:56.628617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:56.628716 (+ 99us) server_negotiation.cc:407] Connection header received
0809 04:56:56.628902 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:56.628905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:56.628996 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:56.629134 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:56.629419 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.630092 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.631429 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.631947 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.632040 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.632469 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:56.632993 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:56.633096 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:56.633197 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 04:56:56.633262 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:56:56.641597 11159 raft_consensus.cc:1261] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Refusing update from remote peer 1b732d802a894d95a62575549c3af76d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:56.641614 10939 raft_consensus.cc:1261] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Refusing update from remote peer 1b732d802a894d95a62575549c3af76d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:56.642551 11286 consensus_queue.cc:1035] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:56.642916 11279 consensus_queue.cc:1035] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:56.654507 11295 mvcc.cc:205] Tried to move back new op lower bound from 7316485802554662912 to 7316485802435194880. Current Snapshot: MvccSnapshot[applied={T|T < 7316485802554662912}]
I20260809 04:56:56.654834 11293 mvcc.cc:205] Tried to move back new op lower bound from 7316485802554662912 to 7316485802435194880. Current Snapshot: MvccSnapshot[applied={T|T < 7316485802554662912}]
I20260809 04:56:56.667915 10447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:56.668203 10447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:56.671737 10447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607: Sending DeleteTablet for 3 replicas of tablet a568fc239a39445e9a6b5bfec9ddf080
I20260809 04:56:56.673000 10904 tablet_service.cc:1514] Processing DeleteTablet for tablet a568fc239a39445e9a6b5bfec9ddf080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:56 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:36826
I20260809 04:56:56.673003 11126 tablet_service.cc:1514] Processing DeleteTablet for tablet a568fc239a39445e9a6b5bfec9ddf080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:56 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:49686
I20260809 04:56:56.673333 10674 tablet_service.cc:1514] Processing DeleteTablet for tablet a568fc239a39445e9a6b5bfec9ddf080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:56 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:55484
I20260809 04:56:56.675034 10447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:56.675745 10447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:56.683687 11125 tablet_service.cc:1467] Processing CreateTablet for tablet a996df59528c412d9aaa9eb2cf34b880 (DEFAULT_TABLE table=test-table [id=0c3bc3281cad4a57b6c443f68a26d826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:56.683844 10902 tablet_service.cc:1467] Processing CreateTablet for tablet a996df59528c412d9aaa9eb2cf34b880 (DEFAULT_TABLE table=test-table [id=0c3bc3281cad4a57b6c443f68a26d826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:56.684286 11125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a996df59528c412d9aaa9eb2cf34b880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:56.684355 10902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a996df59528c412d9aaa9eb2cf34b880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:56.688369 10673 tablet_service.cc:1467] Processing CreateTablet for tablet a996df59528c412d9aaa9eb2cf34b880 (DEFAULT_TABLE table=test-table [id=0c3bc3281cad4a57b6c443f68a26d826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:56.688923 10673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a996df59528c412d9aaa9eb2cf34b880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:56.690788 11267 tablet_bootstrap.cc:492] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f: Bootstrap starting.
I20260809 04:56:56.692409 11267 tablet_bootstrap.cc:654] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:56.693406 11266 tablet_bootstrap.cc:492] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666: Bootstrap starting.
I20260809 04:56:56.694998 11266 tablet_bootstrap.cc:654] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:56.698752 11268 tablet_bootstrap.cc:492] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d: Bootstrap starting.
I20260809 04:56:56.700529 11268 tablet_bootstrap.cc:654] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:56.703737 11305 tablet_replica.cc:330] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: stopping tablet replica
I20260809 04:56:56.704329 11305 raft_consensus.cc:2229] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:56.704973 11305 raft_consensus.cc:2258] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:56.707297 11305 ts_tablet_manager.cc:1905] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:56.707871 11304 tablet_replica.cc:330] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: stopping tablet replica
I20260809 04:56:56.709050 11304 raft_consensus.cc:2229] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:56.710150 11304 raft_consensus.cc:2258] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:56.711864 11266 tablet_bootstrap.cc:492] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666: No bootstrap required, opened a new log
I20260809 04:56:56.712026 11266 ts_tablet_manager.cc:1397] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.011s
I20260809 04:56:56.712860 11266 raft_consensus.cc:348] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.713089 11266 raft_consensus.cc:374] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:56.715149 11304 ts_tablet_manager.cc:1905] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:56.716611 11268 tablet_bootstrap.cc:492] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d: No bootstrap required, opened a new log
I20260809 04:56:56.716843 11268 ts_tablet_manager.cc:1397] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.002s
I20260809 04:56:56.717779 11268 raft_consensus.cc:348] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.718101 11268 raft_consensus.cc:374] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:56.718251 11268 raft_consensus.cc:729] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b732d802a894d95a62575549c3af76d, State: Initialized, Role: FOLLOWER
I20260809 04:56:56.718681 11268 consensus_queue.cc:260] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.719405 11268 ts_tablet_manager.cc:1428] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 04:56:56.713200 11266 raft_consensus.cc:729] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddde5dc74c934a508fe7228a1d83a666, State: Initialized, Role: FOLLOWER
I20260809 04:56:56.720041 11267 tablet_bootstrap.cc:492] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f: No bootstrap required, opened a new log
I20260809 04:56:56.720187 11267 ts_tablet_manager.cc:1397] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.004s
I20260809 04:56:56.720278 11266 consensus_queue.cc:260] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.720990 11267 raft_consensus.cc:348] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.721212 11267 raft_consensus.cc:374] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:56.721314 11267 raft_consensus.cc:729] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c50f84ec8edc4440a83dc942c1bbd55f, State: Initialized, Role: FOLLOWER
I20260809 04:56:56.721659 11266 ts_tablet_manager.cc:1428] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260809 04:56:56.721606 11267 consensus_queue.cc:260] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.722672 11305 ts_tablet_manager.cc:1918] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:56.722724 11267 ts_tablet_manager.cc:1428] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 04:56:56.722925 11305 log.cc:1196] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/a568fc239a39445e9a6b5bfec9ddf080
I20260809 04:56:56.723392 11307 tablet_replica.cc:330] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: stopping tablet replica
I20260809 04:56:56.723712 11305 ts_tablet_manager.cc:1939] T a568fc239a39445e9a6b5bfec9ddf080 P ddde5dc74c934a508fe7228a1d83a666: Deleting consensus metadata
I20260809 04:56:56.724697 11307 raft_consensus.cc:2229] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:56.725328 11307 raft_consensus.cc:2258] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:56.726140 10428 catalog_manager.cc:4841] TS ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111): tablet a568fc239a39445e9a6b5bfec9ddf080 (table test-table [id=bc01b8162162405794a34203a1359428]) successfully deleted
I20260809 04:56:56.729149 11307 ts_tablet_manager.cc:1905] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:56.731158 11304 ts_tablet_manager.cc:1918] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:56.731335 11304 log.cc:1196] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/a568fc239a39445e9a6b5bfec9ddf080
I20260809 04:56:56.731791 11304 ts_tablet_manager.cc:1939] T a568fc239a39445e9a6b5bfec9ddf080 P c50f84ec8edc4440a83dc942c1bbd55f: Deleting consensus metadata
I20260809 04:56:56.733337 10425 catalog_manager.cc:4841] TS c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195:41889): tablet a568fc239a39445e9a6b5bfec9ddf080 (table test-table [id=bc01b8162162405794a34203a1359428]) successfully deleted
I20260809 04:56:56.737002 11307 ts_tablet_manager.cc:1918] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:56.737188 11307 log.cc:1196] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/a568fc239a39445e9a6b5bfec9ddf080
I20260809 04:56:56.737771 11307 ts_tablet_manager.cc:1939] T a568fc239a39445e9a6b5bfec9ddf080 P 1b732d802a894d95a62575549c3af76d: Deleting consensus metadata
I20260809 04:56:56.739473 10428 catalog_manager.cc:4841] TS 1b732d802a894d95a62575549c3af76d (127.29.243.193:38973): tablet a568fc239a39445e9a6b5bfec9ddf080 (table test-table [id=bc01b8162162405794a34203a1359428]) successfully deleted
I20260809 04:56:56.877760 11286 raft_consensus.cc:482] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:56.878021 11286 raft_consensus.cc:504] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.878898 11286 leader_election.cc:288] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111), c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195:41889)
I20260809 04:56:56.879630 11159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a996df59528c412d9aaa9eb2cf34b880" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" is_pre_election: true
I20260809 04:56:56.879629 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a996df59528c412d9aaa9eb2cf34b880" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddde5dc74c934a508fe7228a1d83a666" is_pre_election: true
I20260809 04:56:56.879904 10939 raft_consensus.cc:2454] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 0.
I20260809 04:56:56.879931 11159 raft_consensus.cc:2454] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 0.
I20260809 04:56:56.880425 10600 leader_election.cc:302] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b732d802a894d95a62575549c3af76d, c50f84ec8edc4440a83dc942c1bbd55f, ddde5dc74c934a508fe7228a1d83a666; no voters:
I20260809 04:56:56.880746 11286 raft_consensus.cc:2781] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:56.880857 11286 raft_consensus.cc:482] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:56.880966 11286 raft_consensus.cc:3037] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:56.884160 11286 raft_consensus.cc:504] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.884858 11286 leader_election.cc:288] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 election: Requested vote from peers ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111), c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195:41889)
I20260809 04:56:56.885304 11159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a996df59528c412d9aaa9eb2cf34b880" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c50f84ec8edc4440a83dc942c1bbd55f"
I20260809 04:56:56.885336 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a996df59528c412d9aaa9eb2cf34b880" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddde5dc74c934a508fe7228a1d83a666"
I20260809 04:56:56.885551 10939 raft_consensus.cc:3037] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:56.885533 11159 raft_consensus.cc:3037] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:56.888019 11159 raft_consensus.cc:2454] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 1.
I20260809 04:56:56.888409 10599 leader_election.cc:302] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b732d802a894d95a62575549c3af76d, c50f84ec8edc4440a83dc942c1bbd55f; no voters:
I20260809 04:56:56.888401 10939 raft_consensus.cc:2454] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 1.
I20260809 04:56:56.888687 11286 raft_consensus.cc:2781] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:56.888860 11286 raft_consensus.cc:686] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 1 LEADER]: Becoming Leader. State: Replica: 1b732d802a894d95a62575549c3af76d, State: Running, Role: LEADER
I20260809 04:56:56.889138 11286 consensus_queue.cc:237] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:56.891844 10448 catalog_manager.cc:5495] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b732d802a894d95a62575549c3af76d (127.29.243.193). New cstate: current_term: 1 leader_uuid: "1b732d802a894d95a62575549c3af76d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } health_report { overall_health: UNKNOWN } } }
Aug 09 04:56:56 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251414, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.254@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:56.972261 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.953750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:37561 (local address 127.0.0.1:34116)
0809 04:56:56.954423 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:56.954448 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:56.954618 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:56.955016 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:56.955021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:56.955754 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:56:56.955992 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.956003 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.957148 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.957154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.958650 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:56.958667 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.959512 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.959520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:56.959703 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.963602 (+ 3899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:56:56.963627 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:56:56.966489 (+ 2862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:56:56.969710 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:56.969724 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:56.969732 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:56.970151 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:56.970508 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:56:56.970513 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:56:56.970517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:56:56.970664 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:56:56.971168 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:56:56.971176 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:56:56.971387 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 04:56:56.971534 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 04:56:56.971811 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":347,"threads_started":1}
I20260809 04:56:56.972337 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:56.953815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34116 (local address 127.29.243.254:37561)
0809 04:56:56.954742 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:56.954747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:56.954765 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:56:56.954805 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:56.954808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:56.954861 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:56.954958 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:56.956124 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.956961 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.958828 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:56.959294 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:56.959409 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:56.966707 (+ 7298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:56.966713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:56.966717 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:56.966751 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:56.969535 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:56.970265 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:56.970269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:56.970271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:56.970397 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:56.970768 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:56.970771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:56.970773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:56.971043 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:56.971174 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:56.971631 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 04:56:56.971958 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":785,"thread_start_us":652,"threads_started":1}
I20260809 04:56:56.978178 10448 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:34116:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:56:56.979292 10448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:56.986703 10674 tablet_service.cc:1467] Processing CreateTablet for tablet 929bcecda678446c8cf45a285d267658 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9cc8f29af854278a964be4350edd6a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:56:56.986727 10904 tablet_service.cc:1467] Processing CreateTablet for tablet 929bcecda678446c8cf45a285d267658 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9cc8f29af854278a964be4350edd6a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:56:56.987360 10674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929bcecda678446c8cf45a285d267658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:56.987192 11126 tablet_service.cc:1467] Processing CreateTablet for tablet 929bcecda678446c8cf45a285d267658 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9cc8f29af854278a964be4350edd6a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:56:56.987473 10904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929bcecda678446c8cf45a285d267658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:56.987794 11126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929bcecda678446c8cf45a285d267658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:56.995028 11267 tablet_bootstrap.cc:492] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: Bootstrap starting.
I20260809 04:56:56.995501 11266 tablet_bootstrap.cc:492] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666: Bootstrap starting.
I20260809 04:56:56.996800 11267 tablet_bootstrap.cc:654] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:56.997061 11266 tablet_bootstrap.cc:654] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:56.997877 11268 tablet_bootstrap.cc:492] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d: Bootstrap starting.
I20260809 04:56:56.999680 11268 tablet_bootstrap.cc:654] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:57.004148 11266 tablet_bootstrap.cc:492] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666: No bootstrap required, opened a new log
I20260809 04:56:57.004331 11266 ts_tablet_manager.cc:1397] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260809 04:56:57.005414 11268 tablet_bootstrap.cc:492] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d: No bootstrap required, opened a new log
I20260809 04:56:57.005614 11268 ts_tablet_manager.cc:1397] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260809 04:56:57.005568 11266 raft_consensus.cc:348] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.005822 11266 raft_consensus.cc:374] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:57.005909 11266 raft_consensus.cc:729] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddde5dc74c934a508fe7228a1d83a666, State: Initialized, Role: FOLLOWER
I20260809 04:56:57.006178 11266 consensus_queue.cc:260] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.006503 11268 raft_consensus.cc:348] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.006763 11268 raft_consensus.cc:374] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:57.006860 11268 raft_consensus.cc:729] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b732d802a894d95a62575549c3af76d, State: Initialized, Role: FOLLOWER
I20260809 04:56:57.006646 11278 tablet_replica.cc:439] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } } }
I20260809 04:56:57.006879 11267 tablet_bootstrap.cc:492] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: No bootstrap required, opened a new log
I20260809 04:56:57.007019 11278 tablet_replica.cc:442] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:57.007230 11267 ts_tablet_manager.cc:1397] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260809 04:56:57.007141 11268 consensus_queue.cc:260] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.007639 11286 tablet_replica.cc:439] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } } }
I20260809 04:56:57.007946 11286 tablet_replica.cc:442] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:57.008070 11266 ts_tablet_manager.cc:1428] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 04:56:57.008374 11267 raft_consensus.cc:348] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.008647 11267 raft_consensus.cc:374] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:57.008721 11267 raft_consensus.cc:729] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c50f84ec8edc4440a83dc942c1bbd55f, State: Initialized, Role: FOLLOWER
I20260809 04:56:57.008951 11267 consensus_queue.cc:260] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.009454 11281 tablet_replica.cc:439] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } } }
I20260809 04:56:57.009778 11281 tablet_replica.cc:442] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:57.009882 11267 ts_tablet_manager.cc:1428] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:56:57.011308 11268 ts_tablet_manager.cc:1428] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260809 04:56:57.112498 11281 raft_consensus.cc:482] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:57.112700 11281 raft_consensus.cc:504] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.114323 11281 leader_election.cc:288] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b732d802a894d95a62575549c3af76d (127.29.243.193:38973), ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111)
I20260809 04:56:57.121902 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.114701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:49465 (local address 127.29.243.193:38973)
0809 04:56:57.114920 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:57.114927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:57.115174 (+ 247us) server_negotiation.cc:407] Connection header received
0809 04:56:57.115425 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:57.115431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:57.115522 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:57.115625 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:57.116508 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.117541 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.119912 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.121002 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.121174 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.121252 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:57.121546 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 04:56:57.121665 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:56:57.122314 11332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.115404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:36549 (local address 127.29.243.194:45111)
0809 04:56:57.115845 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:57.115851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:57.116377 (+ 526us) server_negotiation.cc:407] Connection header received
0809 04:56:57.116447 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:57.116452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:57.116554 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:57.116653 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:56:57.117143 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.118038 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.120207 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.121371 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.121501 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.121578 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:57.121922 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 04:56:57.122071 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":226,"threads_started":1}
I20260809 04:56:57.122313 11331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.115219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:45111 (local address 127.29.243.195:36549)
0809 04:56:57.116131 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:57.116153 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:57.116276 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:57.116694 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:57.116698 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:57.116721 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:57.116984 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:57.116998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.118191 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.118199 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:57.120075 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:57.120086 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.121568 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.121575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:57.121753 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.121787 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 04:56:57.121884 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 04:56:57.122002 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":1,"thread_start_us":648,"threads_started":1}
I20260809 04:56:57.122360 11327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.114599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38973 (local address 127.29.243.195:49465)
0809 04:56:57.115061 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:57.115080 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:57.115229 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:57.116081 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:57.116084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:57.116108 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:56:57.116332 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:57.116345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.117688 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.117692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:57.119745 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:57.119765 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.121171 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.121176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:57.121345 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.121373 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 04:56:57.121509 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 04:56:57.121656 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":22,"thread_start_us":258,"threads_started":1}
I20260809 04:56:57.122965 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929bcecda678446c8cf45a285d267658" candidate_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddde5dc74c934a508fe7228a1d83a666" is_pre_election: true
I20260809 04:56:57.123111 10714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929bcecda678446c8cf45a285d267658" candidate_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b732d802a894d95a62575549c3af76d" is_pre_election: true
I20260809 04:56:57.123219 10939 raft_consensus.cc:2454] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c50f84ec8edc4440a83dc942c1bbd55f in term 0.
I20260809 04:56:57.123481 10714 raft_consensus.cc:2454] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c50f84ec8edc4440a83dc942c1bbd55f in term 0.
I20260809 04:56:57.123708 11051 leader_election.cc:302] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c50f84ec8edc4440a83dc942c1bbd55f, ddde5dc74c934a508fe7228a1d83a666; no voters:
I20260809 04:56:57.124104 11281 raft_consensus.cc:2781] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:57.124210 11281 raft_consensus.cc:482] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:57.124307 11281 raft_consensus.cc:3037] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:57.126443 11281 raft_consensus.cc:504] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.127068 11281 leader_election.cc:288] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [CANDIDATE]: Term 1 election: Requested vote from peers 1b732d802a894d95a62575549c3af76d (127.29.243.193:38973), ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111)
I20260809 04:56:57.127562 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929bcecda678446c8cf45a285d267658" candidate_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddde5dc74c934a508fe7228a1d83a666"
I20260809 04:56:57.127574 10714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929bcecda678446c8cf45a285d267658" candidate_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b732d802a894d95a62575549c3af76d"
I20260809 04:56:57.127748 10939 raft_consensus.cc:3037] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:57.127775 10714 raft_consensus.cc:3037] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:57.129666 10939 raft_consensus.cc:2454] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c50f84ec8edc4440a83dc942c1bbd55f in term 1.
I20260809 04:56:57.129971 10714 raft_consensus.cc:2454] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c50f84ec8edc4440a83dc942c1bbd55f in term 1.
I20260809 04:56:57.130019 11051 leader_election.cc:302] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c50f84ec8edc4440a83dc942c1bbd55f, ddde5dc74c934a508fe7228a1d83a666; no voters:
I20260809 04:56:57.130270 11281 raft_consensus.cc:2781] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:57.130618 11281 raft_consensus.cc:686] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 LEADER]: Becoming Leader. State: Replica: c50f84ec8edc4440a83dc942c1bbd55f, State: Running, Role: LEADER
I20260809 04:56:57.131029 11281 consensus_queue.cc:237] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.133076 11333 tablet_replica.cc:439] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: TxnStatusTablet state changed. Reason: New leader c50f84ec8edc4440a83dc942c1bbd55f. Latest consensus state: current_term: 1 leader_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } } }
I20260809 04:56:57.133491 11333 tablet_replica.cc:442] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:56:57.134780 10447 catalog_manager.cc:5495] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f reported cstate change: term changed from 0 to 1, leader changed from <none> to c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195). New cstate: current_term: 1 leader_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } health_report { overall_health: HEALTHY } } }
I20260809 04:56:57.140213 11337 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:56:57.140448 11337 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:56:57.163553 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.154998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41889 (local address 127.0.0.1:46274)
0809 04:56:57.155300 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:57.155324 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:57.155479 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:57.158387 (+ 2908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:57.158392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:57.158405 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:56:57.158617 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:57.158632 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.159931 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.159937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:57.160932 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:57.160944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.161678 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.161684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:57.161889 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.161964 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:56:57.162971 (+ 1007us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:56:57.162993 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:56:57.163066 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 04:56:57.163220 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 04:56:57.163563 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.154932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46274 (local address 127.29.243.195:41889)
0809 04:56:57.157995 (+ 3063us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:57.157999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:57.158019 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:56:57.158092 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:57.158096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:57.158175 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:57.158565 (+ 390us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:57.158849 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.159798 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.161076 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.161525 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.161634 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.162176 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:57.162844 (+ 668us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:57.163047 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:57.163098 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 04:56:57.163175 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2968,"thread_start_us":2821,"threads_started":1}
I20260809 04:56:57.168186 10939 raft_consensus.cc:1261] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Refusing update from remote peer c50f84ec8edc4440a83dc942c1bbd55f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:57.168814 10714 raft_consensus.cc:1261] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Refusing update from remote peer c50f84ec8edc4440a83dc942c1bbd55f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:57.169279 11333 consensus_queue.cc:1035] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:57.169584 11281 consensus_queue.cc:1035] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:57.174885 11278 tablet_replica.cc:439] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666: TxnStatusTablet state changed. Reason: New leader c50f84ec8edc4440a83dc942c1bbd55f. Latest consensus state: current_term: 1 leader_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } } }
I20260809 04:56:57.175230 11278 tablet_replica.cc:442] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:57.180833 11344 mvcc.cc:205] Tried to move back new op lower bound from 7316485804713385984 to 7316485804572610560. Current Snapshot: MvccSnapshot[applied={T|T < 7316485804713385984}]
I20260809 04:56:57.181308 11286 tablet_replica.cc:439] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d: TxnStatusTablet state changed. Reason: New leader c50f84ec8edc4440a83dc942c1bbd55f. Latest consensus state: current_term: 1 leader_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } } }
I20260809 04:56:57.181541 11286 tablet_replica.cc:442] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:57.182185 11281 tablet_replica.cc:439] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } } }
I20260809 04:56:57.182471 11281 tablet_replica.cc:442] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:56:57.182729 11281 tablet_replica.cc:439] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } } }
I20260809 04:56:57.182940 11281 tablet_replica.cc:442] T 929bcecda678446c8cf45a285d267658 P c50f84ec8edc4440a83dc942c1bbd55f: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:56:57.190377 11278 tablet_replica.cc:439] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } } }
I20260809 04:56:57.190631 11278 tablet_replica.cc:442] T 929bcecda678446c8cf45a285d267658 P ddde5dc74c934a508fe7228a1d83a666: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:57.191861 11286 tablet_replica.cc:439] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } } }
I20260809 04:56:57.192112 11286 tablet_replica.cc:442] T 929bcecda678446c8cf45a285d267658 P 1b732d802a894d95a62575549c3af76d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:56:57.207885 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.202523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:41087 (local address 127.29.243.195:41889)
0809 04:56:57.202673 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:57.202679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:57.203232 (+ 553us) server_negotiation.cc:407] Connection header received
0809 04:56:57.203478 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:57.203484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:57.203555 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:57.203635 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:57.204129 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.204898 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.206188 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.206642 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.206751 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.207165 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:57.207300 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:57.207389 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:57.207512 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 04:56:57.207591 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:56:57.207990 11360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.202495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41889 (local address 127.29.243.193:41087)
0809 04:56:57.203123 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:57.203138 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:57.203331 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:57.203662 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:57.203667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:57.203678 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:56:57.203988 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:57.203997 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.205032 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.205036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:57.206050 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:57.206063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.206790 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.206796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:57.206936 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.206999 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:56:57.207391 (+ 392us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:56:57.207403 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 04:56:57.207475 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 04:56:57.207623 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":300,"threads_started":1}
I20260809 04:56:57.217329 10939 raft_consensus.cc:1261] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Refusing update from remote peer 1b732d802a894d95a62575549c3af76d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:57.217398 11159 raft_consensus.cc:1261] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Refusing update from remote peer 1b732d802a894d95a62575549c3af76d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:57.217900 11279 consensus_queue.cc:1035] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:57.218374 11286 consensus_queue.cc:1035] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:57.256563 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.247167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:52313 (local address 127.29.243.193:38973)
0809 04:56:57.247421 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:57.247427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:57.248771 (+ 1344us) server_negotiation.cc:407] Connection header received
0809 04:56:57.249056 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:57.249078 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:57.249173 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:57.249253 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:57.250618 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.251712 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.254269 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.254940 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.255037 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.255755 (+ 718us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:57.255888 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:57.255949 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:57.256177 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 04:56:57.256263 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260809 04:56:57.256939 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.247221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:38973 (local address 127.29.243.195:52313)
0809 04:56:57.248667 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0809 04:56:57.248692 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 04:56:57.248898 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:56:57.249634 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:56:57.249639 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:56:57.249652 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:56:57.250411 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:57.250427 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.252095 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.252101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:57.254086 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:56:57.254104 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.255262 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.255288 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:56:57.255479 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.255549 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:56:57.256033 (+ 484us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:56:57.256054 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:56:57.256134 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:56:57.256268 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1214,"thread_start_us":1038,"threads_started":1}
I20260809 04:56:57.297629 10446 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:57.297855 10446 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:57.300910 10446 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607: Sending DeleteTablet for 3 replicas of tablet a996df59528c412d9aaa9eb2cf34b880
I20260809 04:56:57.301767 11126 tablet_service.cc:1514] Processing DeleteTablet for tablet a996df59528c412d9aaa9eb2cf34b880 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:57 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:49686
I20260809 04:56:57.301807 10904 tablet_service.cc:1514] Processing DeleteTablet for tablet a996df59528c412d9aaa9eb2cf34b880 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:57 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:36826
I20260809 04:56:57.301970 10674 tablet_service.cc:1514] Processing DeleteTablet for tablet a996df59528c412d9aaa9eb2cf34b880 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:57 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:55484
I20260809 04:56:57.311911 11374 tablet_replica.cc:330] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d: stopping tablet replica
I20260809 04:56:57.312074 11371 tablet_replica.cc:330] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666: stopping tablet replica
I20260809 04:56:57.312155 11374 raft_consensus.cc:2229] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:57.312265 11371 raft_consensus.cc:2229] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:57.312355 11373 tablet_replica.cc:330] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f: stopping tablet replica
I20260809 04:56:57.312523 11374 raft_consensus.cc:2258] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:57.312554 11371 raft_consensus.cc:2258] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:57.312568 11373 raft_consensus.cc:2229] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:57.313663 11371 ts_tablet_manager.cc:1905] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:57.313663 11374 ts_tablet_manager.cc:1905] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:57.320827 11374 ts_tablet_manager.cc:1918] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:57.320990 11374 log.cc:1196] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/a996df59528c412d9aaa9eb2cf34b880
I20260809 04:56:57.321142 11371 ts_tablet_manager.cc:1918] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:57.321273 11371 log.cc:1196] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/a996df59528c412d9aaa9eb2cf34b880
I20260809 04:56:57.321281 11373 raft_consensus.cc:2258] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:57.321485 11374 ts_tablet_manager.cc:1939] T a996df59528c412d9aaa9eb2cf34b880 P 1b732d802a894d95a62575549c3af76d: Deleting consensus metadata
I20260809 04:56:57.321728 11371 ts_tablet_manager.cc:1939] T a996df59528c412d9aaa9eb2cf34b880 P ddde5dc74c934a508fe7228a1d83a666: Deleting consensus metadata
I20260809 04:56:57.322650 11373 ts_tablet_manager.cc:1905] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:57.322702 10428 catalog_manager.cc:4841] TS ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111): tablet a996df59528c412d9aaa9eb2cf34b880 (table test-table [id=0c3bc3281cad4a57b6c443f68a26d826]) successfully deleted
I20260809 04:56:57.323060 10428 catalog_manager.cc:4841] TS 1b732d802a894d95a62575549c3af76d (127.29.243.193:38973): tablet a996df59528c412d9aaa9eb2cf34b880 (table test-table [id=0c3bc3281cad4a57b6c443f68a26d826]) successfully deleted
I20260809 04:56:57.330519 11373 ts_tablet_manager.cc:1918] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:57.330675 11373 log.cc:1196] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/a996df59528c412d9aaa9eb2cf34b880
I20260809 04:56:57.331148 11373 ts_tablet_manager.cc:1939] T a996df59528c412d9aaa9eb2cf34b880 P c50f84ec8edc4440a83dc942c1bbd55f: Deleting consensus metadata
I20260809 04:56:57.332391 10425 catalog_manager.cc:4841] TS c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195:41889): tablet a996df59528c412d9aaa9eb2cf34b880 (table test-table [id=0c3bc3281cad4a57b6c443f68a26d826]) successfully deleted
Aug 09 04:56:57 dist-test-slave-21b9 krb5kdc[10379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:56:57 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:56:57 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:56:57.371589 11322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:56:57.361059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34132 (local address 127.29.243.254:37561)
0809 04:56:57.361284 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:57.361291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:57.361625 (+ 334us) server_negotiation.cc:407] Connection header received
0809 04:56:57.361787 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:57.361793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:57.361881 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:57.362002 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:57.363426 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.364540 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.365880 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.366553 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.366684 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.371306 (+ 4622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34132: BlockingRecv error: recv got EOF from 127.0.0.1:34132 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 04:56:57 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM
I20260809 04:56:57.389209 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.374237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.29.243.254:37561)
0809 04:56:57.374428 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:57.374434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:57.374748 (+ 314us) server_negotiation.cc:407] Connection header received
0809 04:56:57.374906 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:57.374909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:57.374973 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:57.375060 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:57.376676 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.377396 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.379190 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.379622 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.379716 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.384113 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:57.384118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:57.384121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:57.384156 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:57.386684 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:57.387479 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:57.387503 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:57.387505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:57.387631 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:57.387994 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:57.387997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:57.387999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:57.388245 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:57.388451 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:57.388764 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 04:56:57.388935 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:56:57.392583 10446 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:57.393200 10446 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:57.400074 10674 tablet_service.cc:1467] Processing CreateTablet for tablet 3b77acac9bab4aec90e10e84f8923120 (DEFAULT_TABLE table=test-table [id=256c3ed19e774c99811375af8c2454f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:57.400447 11126 tablet_service.cc:1467] Processing CreateTablet for tablet 3b77acac9bab4aec90e10e84f8923120 (DEFAULT_TABLE table=test-table [id=256c3ed19e774c99811375af8c2454f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:57.400590 10674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b77acac9bab4aec90e10e84f8923120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:57.400879 11126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b77acac9bab4aec90e10e84f8923120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:57.400990 10904 tablet_service.cc:1467] Processing CreateTablet for tablet 3b77acac9bab4aec90e10e84f8923120 (DEFAULT_TABLE table=test-table [id=256c3ed19e774c99811375af8c2454f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:57.401408 10904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b77acac9bab4aec90e10e84f8923120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:57.407436 11267 tablet_bootstrap.cc:492] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f: Bootstrap starting.
I20260809 04:56:57.407436 11268 tablet_bootstrap.cc:492] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d: Bootstrap starting.
I20260809 04:56:57.408442 11266 tablet_bootstrap.cc:492] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666: Bootstrap starting.
I20260809 04:56:57.409138 11268 tablet_bootstrap.cc:654] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:57.409142 11267 tablet_bootstrap.cc:654] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:57.410048 11266 tablet_bootstrap.cc:654] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:57.418278 11266 tablet_bootstrap.cc:492] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666: No bootstrap required, opened a new log
I20260809 04:56:57.418290 11267 tablet_bootstrap.cc:492] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f: No bootstrap required, opened a new log
I20260809 04:56:57.418439 11266 ts_tablet_manager.cc:1397] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260809 04:56:57.418447 11267 ts_tablet_manager.cc:1397] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260809 04:56:57.419260 11267 raft_consensus.cc:348] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.419251 11266 raft_consensus.cc:348] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.419535 11267 raft_consensus.cc:374] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:57.419535 11266 raft_consensus.cc:374] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:57.419626 11267 raft_consensus.cc:729] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c50f84ec8edc4440a83dc942c1bbd55f, State: Initialized, Role: FOLLOWER
I20260809 04:56:57.419631 11266 raft_consensus.cc:729] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddde5dc74c934a508fe7228a1d83a666, State: Initialized, Role: FOLLOWER
I20260809 04:56:57.419847 11266 consensus_queue.cc:260] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.419925 11267 consensus_queue.cc:260] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.420075 11268 tablet_bootstrap.cc:492] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d: No bootstrap required, opened a new log
I20260809 04:56:57.420377 11266 ts_tablet_manager.cc:1428] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 04:56:57.421411 11268 ts_tablet_manager.cc:1397] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260809 04:56:57.422187 11267 ts_tablet_manager.cc:1428] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 04:56:57.422134 11268 raft_consensus.cc:348] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.422425 11268 raft_consensus.cc:374] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:57.422536 11268 raft_consensus.cc:729] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b732d802a894d95a62575549c3af76d, State: Initialized, Role: FOLLOWER
I20260809 04:56:57.422860 11268 consensus_queue.cc:260] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.423488 11268 ts_tablet_manager.cc:1428] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:56:57.709380 11281 raft_consensus.cc:482] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:57.709589 11281 raft_consensus.cc:504] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.710264 11281 leader_election.cc:288] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b732d802a894d95a62575549c3af76d (127.29.243.193:38973), ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111)
I20260809 04:56:57.710765 10714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b77acac9bab4aec90e10e84f8923120" candidate_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b732d802a894d95a62575549c3af76d" is_pre_election: true
I20260809 04:56:57.710807 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b77acac9bab4aec90e10e84f8923120" candidate_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddde5dc74c934a508fe7228a1d83a666" is_pre_election: true
I20260809 04:56:57.711004 10714 raft_consensus.cc:2454] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c50f84ec8edc4440a83dc942c1bbd55f in term 0.
I20260809 04:56:57.711009 10939 raft_consensus.cc:2454] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c50f84ec8edc4440a83dc942c1bbd55f in term 0.
I20260809 04:56:57.711421 11051 leader_election.cc:302] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b732d802a894d95a62575549c3af76d, c50f84ec8edc4440a83dc942c1bbd55f; no voters:
I20260809 04:56:57.711685 11281 raft_consensus.cc:2781] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:57.711792 11281 raft_consensus.cc:482] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:57.711879 11281 raft_consensus.cc:3037] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:57.713882 11281 raft_consensus.cc:504] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.714380 11281 leader_election.cc:288] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [CANDIDATE]: Term 1 election: Requested vote from peers 1b732d802a894d95a62575549c3af76d (127.29.243.193:38973), ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111)
I20260809 04:56:57.714879 10714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b77acac9bab4aec90e10e84f8923120" candidate_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b732d802a894d95a62575549c3af76d"
I20260809 04:56:57.714939 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b77acac9bab4aec90e10e84f8923120" candidate_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddde5dc74c934a508fe7228a1d83a666"
I20260809 04:56:57.715071 10714 raft_consensus.cc:3037] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:57.715157 10939 raft_consensus.cc:3037] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:57.717607 10714 raft_consensus.cc:2454] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c50f84ec8edc4440a83dc942c1bbd55f in term 1.
I20260809 04:56:57.717617 10939 raft_consensus.cc:2454] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c50f84ec8edc4440a83dc942c1bbd55f in term 1.
I20260809 04:56:57.718026 11051 leader_election.cc:302] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b732d802a894d95a62575549c3af76d, c50f84ec8edc4440a83dc942c1bbd55f; no voters:
I20260809 04:56:57.718308 11281 raft_consensus.cc:2781] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:57.718449 11281 raft_consensus.cc:686] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 LEADER]: Becoming Leader. State: Replica: c50f84ec8edc4440a83dc942c1bbd55f, State: Running, Role: LEADER
I20260809 04:56:57.718719 11281 consensus_queue.cc:237] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.721453 10447 catalog_manager.cc:5495] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f reported cstate change: term changed from 0 to 1, leader changed from <none> to c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195). New cstate: current_term: 1 leader_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } health_report { overall_health: HEALTHY } } }
I20260809 04:56:57.790788 11398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:57.786053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46286 (local address 127.29.243.195:41889)
0809 04:56:57.786524 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:57.786535 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:57.786555 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:56:57.786623 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:57.786627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:57.786690 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:57.786770 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:57.787099 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.787859 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.789222 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:57.789628 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:57.789720 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:57.790170 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:57.790271 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:57.790333 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:57.790452 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 04:56:57.790517 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":223,"threads_started":1}
I20260809 04:56:57.794080 10714 raft_consensus.cc:1261] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Refusing update from remote peer c50f84ec8edc4440a83dc942c1bbd55f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:57.794219 10939 raft_consensus.cc:1261] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Refusing update from remote peer c50f84ec8edc4440a83dc942c1bbd55f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:57.794667 11281 consensus_queue.cc:1035] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:57.794927 11365 consensus_queue.cc:1035] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:57.812111 10446 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:57.812337 10446 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:57.815141 10446 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607: Sending DeleteTablet for 3 replicas of tablet 3b77acac9bab4aec90e10e84f8923120
I20260809 04:56:57.815975 10674 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b77acac9bab4aec90e10e84f8923120 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:57 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:55484
I20260809 04:56:57.816260 11126 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b77acac9bab4aec90e10e84f8923120 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:57 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:49686
I20260809 04:56:57.816285 10904 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b77acac9bab4aec90e10e84f8923120 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:57 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:36826
I20260809 04:56:57.816581 11374 tablet_replica.cc:330] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d: stopping tablet replica
I20260809 04:56:57.816682 11371 tablet_replica.cc:330] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666: stopping tablet replica
I20260809 04:56:57.816797 11374 raft_consensus.cc:2229] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:57.816903 11371 raft_consensus.cc:2229] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:57.817030 11374 raft_consensus.cc:2258] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:57.817128 11371 raft_consensus.cc:2258] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:57.817396 11373 tablet_replica.cc:330] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f: stopping tablet replica
I20260809 04:56:57.817607 11373 raft_consensus.cc:2229] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:57.818120 11373 raft_consensus.cc:2258] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:57.818557 11371 ts_tablet_manager.cc:1905] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:57.818938 11374 ts_tablet_manager.cc:1905] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:57.820139 11373 ts_tablet_manager.cc:1905] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:57.823865 10446 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:56:57.824581 10446 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:56:57.825174 11371 ts_tablet_manager.cc:1918] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:57.825174 11374 ts_tablet_manager.cc:1918] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:57.825314 11371 log.cc:1196] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/3b77acac9bab4aec90e10e84f8923120
I20260809 04:56:57.825316 11374 log.cc:1196] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/3b77acac9bab4aec90e10e84f8923120
I20260809 04:56:57.825702 11371 ts_tablet_manager.cc:1939] T 3b77acac9bab4aec90e10e84f8923120 P ddde5dc74c934a508fe7228a1d83a666: Deleting consensus metadata
I20260809 04:56:57.826797 10428 catalog_manager.cc:4841] TS ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111): tablet 3b77acac9bab4aec90e10e84f8923120 (table test-table [id=256c3ed19e774c99811375af8c2454f0]) successfully deleted
I20260809 04:56:57.827169 11374 ts_tablet_manager.cc:1939] T 3b77acac9bab4aec90e10e84f8923120 P 1b732d802a894d95a62575549c3af76d: Deleting consensus metadata
I20260809 04:56:57.828510 10428 catalog_manager.cc:4841] TS 1b732d802a894d95a62575549c3af76d (127.29.243.193:38973): tablet 3b77acac9bab4aec90e10e84f8923120 (table test-table [id=256c3ed19e774c99811375af8c2454f0]) successfully deleted
I20260809 04:56:57.830118 11373 ts_tablet_manager.cc:1918] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:56:57.830780 11373 log.cc:1196] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/3b77acac9bab4aec90e10e84f8923120
I20260809 04:56:57.831341 11373 ts_tablet_manager.cc:1939] T 3b77acac9bab4aec90e10e84f8923120 P c50f84ec8edc4440a83dc942c1bbd55f: Deleting consensus metadata
I20260809 04:56:57.832518 10425 catalog_manager.cc:4841] TS c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195:41889): tablet 3b77acac9bab4aec90e10e84f8923120 (table test-table [id=256c3ed19e774c99811375af8c2454f0]) successfully deleted
I20260809 04:56:57.832692 10674 tablet_service.cc:1467] Processing CreateTablet for tablet 41064908cbb8410c917c3c1c66a8c8cf (DEFAULT_TABLE table=test-table [id=2e8ceaf93e9d4ddcb45a7e75066e3ef9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:57.833117 10674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41064908cbb8410c917c3c1c66a8c8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:57.833330 11126 tablet_service.cc:1467] Processing CreateTablet for tablet 41064908cbb8410c917c3c1c66a8c8cf (DEFAULT_TABLE table=test-table [id=2e8ceaf93e9d4ddcb45a7e75066e3ef9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:57.833725 11126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41064908cbb8410c917c3c1c66a8c8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:57.837139 10904 tablet_service.cc:1467] Processing CreateTablet for tablet 41064908cbb8410c917c3c1c66a8c8cf (DEFAULT_TABLE table=test-table [id=2e8ceaf93e9d4ddcb45a7e75066e3ef9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:56:57.837556 10904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41064908cbb8410c917c3c1c66a8c8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:56:57.840387 11267 tablet_bootstrap.cc:492] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f: Bootstrap starting.
I20260809 04:56:57.842067 11267 tablet_bootstrap.cc:654] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:57.843034 11266 tablet_bootstrap.cc:492] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666: Bootstrap starting.
I20260809 04:56:57.843327 11268 tablet_bootstrap.cc:492] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d: Bootstrap starting.
I20260809 04:56:57.844682 11266 tablet_bootstrap.cc:654] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:57.844890 11268 tablet_bootstrap.cc:654] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d: Neither blocks nor log segments found. Creating new log.
I20260809 04:56:57.852617 11268 tablet_bootstrap.cc:492] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d: No bootstrap required, opened a new log
I20260809 04:56:57.852799 11268 ts_tablet_manager.cc:1397] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 04:56:57.853076 11266 tablet_bootstrap.cc:492] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666: No bootstrap required, opened a new log
I20260809 04:56:57.853288 11266 ts_tablet_manager.cc:1397] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.008s
I20260809 04:56:57.853617 11268 raft_consensus.cc:348] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.853883 11268 raft_consensus.cc:374] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:57.853981 11267 tablet_bootstrap.cc:492] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f: No bootstrap required, opened a new log
I20260809 04:56:57.853987 11268 raft_consensus.cc:729] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b732d802a894d95a62575549c3af76d, State: Initialized, Role: FOLLOWER
I20260809 04:56:57.854151 11267 ts_tablet_manager.cc:1397] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.006s
I20260809 04:56:57.854260 11268 consensus_queue.cc:260] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.854336 11266 raft_consensus.cc:348] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.854699 11266 raft_consensus.cc:374] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:57.854820 11266 raft_consensus.cc:729] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddde5dc74c934a508fe7228a1d83a666, State: Initialized, Role: FOLLOWER
I20260809 04:56:57.854925 11267 raft_consensus.cc:348] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.855181 11267 raft_consensus.cc:374] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:56:57.855299 11267 raft_consensus.cc:729] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c50f84ec8edc4440a83dc942c1bbd55f, State: Initialized, Role: FOLLOWER
I20260809 04:56:57.855198 11266 consensus_queue.cc:260] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.855571 11267 consensus_queue.cc:260] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:57.856204 11267 ts_tablet_manager.cc:1428] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 04:56:57.856703 11266 ts_tablet_manager.cc:1428] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 04:56:57.857214 11268 ts_tablet_manager.cc:1428] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 04:56:58.161363 11279 raft_consensus.cc:482] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:56:58.161600 11279 raft_consensus.cc:504] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:58.162333 11279 leader_election.cc:288] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111), c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195:41889)
I20260809 04:56:58.163106 11159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41064908cbb8410c917c3c1c66a8c8cf" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" is_pre_election: true
I20260809 04:56:58.163089 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41064908cbb8410c917c3c1c66a8c8cf" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddde5dc74c934a508fe7228a1d83a666" is_pre_election: true
I20260809 04:56:58.163393 10939 raft_consensus.cc:2454] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 0.
I20260809 04:56:58.163441 11159 raft_consensus.cc:2454] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 0.
I20260809 04:56:58.164090 10600 leader_election.cc:302] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b732d802a894d95a62575549c3af76d, ddde5dc74c934a508fe7228a1d83a666; no voters:
I20260809 04:56:58.164506 11279 raft_consensus.cc:2781] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:56:58.164616 11279 raft_consensus.cc:482] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:56:58.164724 11279 raft_consensus.cc:3037] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:58.167915 11279 raft_consensus.cc:504] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:58.169019 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41064908cbb8410c917c3c1c66a8c8cf" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddde5dc74c934a508fe7228a1d83a666"
I20260809 04:56:58.169040 11159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41064908cbb8410c917c3c1c66a8c8cf" candidate_uuid: "1b732d802a894d95a62575549c3af76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c50f84ec8edc4440a83dc942c1bbd55f"
I20260809 04:56:58.169245 10939 raft_consensus.cc:3037] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:58.169252 11159 raft_consensus.cc:3037] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:56:58.172354 11159 raft_consensus.cc:2454] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 1.
I20260809 04:56:58.172390 10939 raft_consensus.cc:2454] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b732d802a894d95a62575549c3af76d in term 1.
I20260809 04:56:58.172875 10599 leader_election.cc:302] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b732d802a894d95a62575549c3af76d, c50f84ec8edc4440a83dc942c1bbd55f; no voters:
I20260809 04:56:58.173408 11279 leader_election.cc:288] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [CANDIDATE]: Term 1 election: Requested vote from peers ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111), c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195:41889)
I20260809 04:56:58.173691 11279 raft_consensus.cc:2781] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:56:58.173892 11279 raft_consensus.cc:686] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 1 LEADER]: Becoming Leader. State: Replica: 1b732d802a894d95a62575549c3af76d, State: Running, Role: LEADER
I20260809 04:56:58.174202 11279 consensus_queue.cc:237] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } }
I20260809 04:56:58.177242 10447 catalog_manager.cc:5495] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b732d802a894d95a62575549c3af76d (127.29.243.193). New cstate: current_term: 1 leader_uuid: "1b732d802a894d95a62575549c3af76d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b732d802a894d95a62575549c3af76d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 38973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 } health_report { overall_health: UNKNOWN } } }
I20260809 04:56:58.218194 11425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:58.212918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38816 (local address 127.29.243.193:38973)
0809 04:56:58.213856 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:58.213861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:58.213878 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:56:58.213938 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:58.213941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:58.214001 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:58.214074 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:56:58.214473 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:58.215220 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:58.216558 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:58.216982 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:58.217081 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:58.217488 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:56:58.217601 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:56:58.217672 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:58.217840 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 04:56:58.217917 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":695,"threads_started":1}
I20260809 04:56:58.228823 11159 raft_consensus.cc:1261] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Refusing update from remote peer 1b732d802a894d95a62575549c3af76d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:58.229034 10939 raft_consensus.cc:1261] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Refusing update from remote peer 1b732d802a894d95a62575549c3af76d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:56:58.229598 11279 consensus_queue.cc:1035] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c50f84ec8edc4440a83dc942c1bbd55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:58.229874 11420 consensus_queue.cc:1035] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddde5dc74c934a508fe7228a1d83a666" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:56:58.288548 10447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:58.288841 10447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:56:58.292325 10447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41b97b4693254b2b9541fc8144ab8607: Sending DeleteTablet for 3 replicas of tablet 41064908cbb8410c917c3c1c66a8c8cf
I20260809 04:56:58.293109 10674 tablet_service.cc:1514] Processing DeleteTablet for tablet 41064908cbb8410c917c3c1c66a8c8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:58 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:55484
I20260809 04:56:58.293442 11374 tablet_replica.cc:330] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d: stopping tablet replica
I20260809 04:56:58.293448 11126 tablet_service.cc:1514] Processing DeleteTablet for tablet 41064908cbb8410c917c3c1c66a8c8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:58 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:49686
I20260809 04:56:58.293637 11374 raft_consensus.cc:2229] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:56:58.293709 10904 tablet_service.cc:1514] Processing DeleteTablet for tablet 41064908cbb8410c917c3c1c66a8c8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:56:58 UTC) from {username='oryx', principal='oryx/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:36826
I20260809 04:56:58.294028 11371 tablet_replica.cc:330] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666: stopping tablet replica
I20260809 04:56:58.294003 11374 raft_consensus.cc:2258] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:58.294207 11371 raft_consensus.cc:2229] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:58.294305 11373 tablet_replica.cc:330] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f: stopping tablet replica
I20260809 04:56:58.294426 11371 raft_consensus.cc:2258] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:58.294477 11373 raft_consensus.cc:2229] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:56:58.294682 11373 raft_consensus.cc:2258] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:56:58.295635 11371 ts_tablet_manager.cc:1905] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:58.297610 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 10565
I20260809 04:56:58.298062 11374 ts_tablet_manager.cc:1905] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:56:58.302322 11371 ts_tablet_manager.cc:1918] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:58.302465 11371 log.cc:1196] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/41064908cbb8410c917c3c1c66a8c8cf
I20260809 04:56:58.302873 11371 ts_tablet_manager.cc:1939] T 41064908cbb8410c917c3c1c66a8c8cf P ddde5dc74c934a508fe7228a1d83a666: Deleting consensus metadata
I20260809 04:56:58.304018 10428 catalog_manager.cc:4841] TS ddde5dc74c934a508fe7228a1d83a666 (127.29.243.194:45111): tablet 41064908cbb8410c917c3c1c66a8c8cf (table test-table [id=2e8ceaf93e9d4ddcb45a7e75066e3ef9]) successfully deleted
I20260809 04:56:58.304877 11374 ts_tablet_manager.cc:1918] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:58.305022 11374 log.cc:1196] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/41064908cbb8410c917c3c1c66a8c8cf
I20260809 04:56:58.305438 11374 ts_tablet_manager.cc:1939] T 41064908cbb8410c917c3c1c66a8c8cf P 1b732d802a894d95a62575549c3af76d: Deleting consensus metadata
I20260809 04:56:58.306514 10428 catalog_manager.cc:4841] TS 1b732d802a894d95a62575549c3af76d (127.29.243.193:38973): tablet 41064908cbb8410c917c3c1c66a8c8cf (table test-table [id=2e8ceaf93e9d4ddcb45a7e75066e3ef9]) successfully deleted
I20260809 04:56:58.307320 11373 ts_tablet_manager.cc:1905] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 04:56:58 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.243.193@KRBTEST.COM
I20260809 04:56:58.313642 11373 ts_tablet_manager.cc:1918] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:56:58.313767 11373 log.cc:1196] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/41064908cbb8410c917c3c1c66a8c8cf
I20260809 04:56:58.314122 11373 ts_tablet_manager.cc:1939] T 41064908cbb8410c917c3c1c66a8c8cf P c50f84ec8edc4440a83dc942c1bbd55f: Deleting consensus metadata
I20260809 04:56:58.315129 10425 catalog_manager.cc:4841] TS c50f84ec8edc4440a83dc942c1bbd55f (127.29.243.195:41889): tablet 41064908cbb8410c917c3c1c66a8c8cf (table test-table [id=2e8ceaf93e9d4ddcb45a7e75066e3ef9]) successfully deleted
I20260809 04:56:58.318565 11425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:58.300286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38826 (local address 127.29.243.193:38973)
0809 04:56:58.300485 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:58.300492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:58.300933 (+ 441us) server_negotiation.cc:407] Connection header received
0809 04:56:58.301119 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:58.301124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:58.301202 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:58.301297 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:58.302568 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:58.303535 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:58.305296 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:58.305916 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:58.306049 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:58.311449 (+ 5400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:58.311485 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:58.311511 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:58.311556 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:58.315777 (+ 4221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:58.316496 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:58.316508 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:58.316511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:58.316649 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:58.317023 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:58.317029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:58.317033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:58.317404 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:58.317751 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:58.317997 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 04:56:58.318210 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:56:58.319219 10780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:58.451874 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 10565
I20260809 04:56:58.485993 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 10800
Aug 09 04:56:58 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.243.194@KRBTEST.COM
I20260809 04:56:58.502439 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:58.486773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49310 (local address 127.29.243.194:45111)
0809 04:56:58.487234 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:58.487239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:58.487256 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:56:58.487341 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:58.487345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:58.487421 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:58.487514 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:58.488721 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:58.489431 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:58.491252 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:58.491726 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:58.491825 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:58.496770 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:58.496787 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:58.496806 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:58.496853 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:58.499715 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:58.500476 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:58.500484 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:58.500486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:58.500611 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:58.500953 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:58.500955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:58.500957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:58.501303 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:58.501519 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:58.501935 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 04:56:58.502112 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":236,"threads_started":1}
I20260809 04:56:58.503142 11017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:58.601938 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 10800
I20260809 04:56:58.628491 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 11027
Aug 09 04:56:58 dist-test-slave-21b9 krb5kdc[10379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.243.195@KRBTEST.COM
I20260809 04:56:58.643724 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:56:58.629288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46290 (local address 127.29.243.195:41889)
0809 04:56:58.629826 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 04:56:58.629838 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:56:58.629856 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:56:58.629915 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:56:58.629919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:56:58.629988 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:56:58.630092 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:56:58.631401 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:58.632115 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:58.633438 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:56:58.633856 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:56:58.633956 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:56:58.638484 (+ 4528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:56:58.638500 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:56:58.638520 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:56:58.638552 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:56:58.641276 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:58.641962 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:58.641970 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:58.641972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:58.642109 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:56:58.642413 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:56:58.642416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:56:58.642418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:56:58.642762 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:56:58.642997 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 04:56:58.643214 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 04:56:58.643427 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":299,"threads_started":1}
I20260809 04:56:58.644371 11224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:58.787817 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 11027
I20260809 04:56:58.821022 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 10395
I20260809 04:56:58.822330 10509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:56:58.941198 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 10395
2026-08-09T04:56:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6392 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:56:59 dist-test-slave-21b9 krb5kdc[11445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:56:59 dist-test-slave-21b9 krb5kdc[11445](info): set up 2 sockets
Aug 09 04:56:59 dist-test-slave-21b9 krb5kdc[11445](info): commencing operation
krb5kdc: starting...
W20260809 04:57:00.176204 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:57:00 dist-test-slave-21b9 krb5kdc[11445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:57:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:57:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:57:00.441267 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44119
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:44119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:57:00.784451 11464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:00.784737 11464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:00.784871 11464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:00.792263 11464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:57:00.792363 11464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:00.792455 11464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:00.792527 11464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:57:00.792590 11464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:57:00.802439 11464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:44119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:00.803790 11464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:00.805143 11464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:00.817111 11473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:00.817159 11475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:00.817437 11478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:00.818035 11464 server_base.cc:1034] running on GCE node
I20260809 04:57:00.818758 11464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:00.820188 11464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:00.821419 11464 hybrid_clock.cc:648] HybridClock initialized: now 1786251420821343 us; error 58 us; skew 500 ppm
I20260809 04:57:00.821808 11464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T04:57:00Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1941 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:57:01 dist-test-slave-21b9 krb5kdc[11486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:57:01 dist-test-slave-21b9 krb5kdc[11486](info): set up 2 sockets
Aug 09 04:57:01 dist-test-slave-21b9 krb5kdc[11486](info): commencing operation
krb5kdc: starting...
W20260809 04:57:02.116297 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.007s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:57:02 dist-test-slave-21b9 krb5kdc[11486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:57:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:57:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:57:02.376690 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35297
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:35297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:57:02.710369 11505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:02.710664 11505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:02.710781 11505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:02.718852 11505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:57:02.718979 11505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:02.719061 11505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:02.719148 11505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:57:02.719210 11505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:57:02.729301 11505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:35297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:02.730698 11505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:02.732086 11505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:02.743875 11510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:02.743945 11511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:02.743918 11516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:02.744889 11505 server_base.cc:1034] running on GCE node
I20260809 04:57:02.745707 11505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:02.747092 11505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:02.748306 11505 hybrid_clock.cc:648] HybridClock initialized: now 1786251422748254 us; error 34 us; skew 500 ppm
I20260809 04:57:02.748672 11505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:02 dist-test-slave-21b9 krb5kdc[11486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251422, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:02.755831 11505 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:57:02.757388 11505 webserver.cc:460] Webserver started at http://127.29.243.254:38015/ using document root <none> and password file <none>
I20260809 04:57:02.758090 11505 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:02.758193 11505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:02.758461 11505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:02.761476 11505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "0ec96aba9263448aad6314c8533e8026"
format_stamp: "Formatted at 2026-08-09 04:57:02 on dist-test-slave-21b9"
server_key: "9759fd7e03e8c2bb604dcbe9c86e0a37"
server_key_iv: "d73bb2962596a7ad2981d5c3f9fb4f15"
server_key_version: "encryptionkey@0"
I20260809 04:57:02.762179 11505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "0ec96aba9263448aad6314c8533e8026"
format_stamp: "Formatted at 2026-08-09 04:57:02 on dist-test-slave-21b9"
server_key: "9759fd7e03e8c2bb604dcbe9c86e0a37"
server_key_iv: "d73bb2962596a7ad2981d5c3f9fb4f15"
server_key_version: "encryptionkey@0"
I20260809 04:57:02.767318 11505 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260809 04:57:02.770936 11528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:02.772320 11505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:57:02.772512 11505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "0ec96aba9263448aad6314c8533e8026"
format_stamp: "Formatted at 2026-08-09 04:57:02 on dist-test-slave-21b9"
server_key: "9759fd7e03e8c2bb604dcbe9c86e0a37"
server_key_iv: "d73bb2962596a7ad2981d5c3f9fb4f15"
server_key_version: "encryptionkey@0"
I20260809 04:57:02.772686 11505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:02.792999 11505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:02.797044 11505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:02.797282 11505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:02.814980 11505 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:35297
I20260809 04:57:02.814999 11606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:35297 every 8 connection(s)
I20260809 04:57:02.816375 11505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:57:02.820111 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 11505
I20260809 04:57:02.820355 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:57:02.820479 11608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:57:02.821167 30671 external_mini_cluster.cc:1428] Setting key bd73d75429c2e8914a67e1c3e244201d
I20260809 04:57:02.834214 11608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026: Bootstrap starting.
Aug 09 04:57:02 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:57:02.837999 11608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026: Neither blocks nor log segments found. Creating new log.
I20260809 04:57:02.839414 11608 log.cc:824] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026: Log is configured to *not* fsync() on all Append() calls
I20260809 04:57:02.842630 11608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026: No bootstrap required, opened a new log
I20260809 04:57:02.845639 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:02.823309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36196 (local address 127.29.243.254:35297)
0809 04:57:02.824142 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:02.824153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:02.824196 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:57:02.826531 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:02.826553 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:02.826952 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:02.827295 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:02.829058 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:02.830354 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:02.832150 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:02.832801 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:02.832944 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:02.837403 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:02.837413 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:02.837432 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:02.837475 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:02.841099 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:02.841784 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:02.841792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:02.841800 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:02.842013 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:02.842438 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:02.842443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:02.842446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:02.843162 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:02.843478 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:02.843888 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 04:57:02.844307 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":364,"threads_started":1}
I20260809 04:57:02.851003 11608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } }
I20260809 04:57:02.851357 11608 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:57:02.851466 11608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec96aba9263448aad6314c8533e8026, State: Initialized, Role: FOLLOWER
I20260809 04:57:02.851996 11608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } }
I20260809 04:57:02.852218 11608 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:57:02.852347 11608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:57:02.852506 11608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:57:02.854655 11608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } }
I20260809 04:57:02.855135 11608 leader_election.cc:302] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ec96aba9263448aad6314c8533e8026; no voters:
I20260809 04:57:02.855710 11608 leader_election.cc:288] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:57:02.855844 11614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:57:02.856446 11614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 1 LEADER]: Becoming Leader. State: Replica: 0ec96aba9263448aad6314c8533e8026, State: Running, Role: LEADER
I20260809 04:57:02.857017 11614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } }
I20260809 04:57:02.857620 11608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:57:02.859892 11615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ec96aba9263448aad6314c8533e8026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } } }
I20260809 04:57:02.860244 11615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [sys.catalog]: This master's current role is: LEADER
I20260809 04:57:02.863739 11616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ec96aba9263448aad6314c8533e8026. Latest consensus state: current_term: 1 leader_uuid: "0ec96aba9263448aad6314c8533e8026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } } }
I20260809 04:57:02.864262 11616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [sys.catalog]: This master's current role is: LEADER
I20260809 04:57:02.864631 11628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:57:02.869964 11628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:57:02.879552 11628 catalog_manager.cc:1347] Generated new cluster ID: 40b155a017004377995a9f7fd576fc0e
I20260809 04:57:02.879683 11628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:57:02.899832 11628 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:57:02.901629 11628 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:57:02.912587 11628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026: Generated new TSK 0
I20260809 04:57:02.913637 11628 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:57:02.978698 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35297
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:57:03.319532 11648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:03.319824 11648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:03.319968 11648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:03.327090 11648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:03.327212 11648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:03.327386 11648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:57:03.337144 11648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:03.338421 11648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:03.339903 11648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:03.352629 11662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:03.352694 11660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:03.352718 11659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:03.353461 11648 server_base.cc:1034] running on GCE node
I20260809 04:57:03.354082 11648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:03.355002 11648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:03.356213 11648 hybrid_clock.cc:648] HybridClock initialized: now 1786251423356142 us; error 63 us; skew 500 ppm
I20260809 04:57:03.356526 11648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:03 dist-test-slave-21b9 krb5kdc[11486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:03.359642 11648 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:57:03.361001 11648 webserver.cc:460] Webserver started at http://127.29.243.193:36763/ using document root <none> and password file <none>
I20260809 04:57:03.361676 11648 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:03.361792 11648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:03.362099 11648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:03.365154 11648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "af7a1665f46d4388835b89de2b3975cb"
format_stamp: "Formatted at 2026-08-09 04:57:03 on dist-test-slave-21b9"
server_key: "5f6bf1882b5fb59b3a3350b80a85213d"
server_key_iv: "53541fee15d246fc8f3a742f327dbd34"
server_key_version: "encryptionkey@0"
I20260809 04:57:03.365917 11648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "af7a1665f46d4388835b89de2b3975cb"
format_stamp: "Formatted at 2026-08-09 04:57:03 on dist-test-slave-21b9"
server_key: "5f6bf1882b5fb59b3a3350b80a85213d"
server_key_iv: "53541fee15d246fc8f3a742f327dbd34"
server_key_version: "encryptionkey@0"
I20260809 04:57:03.370887 11648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 04:57:03.374297 11673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:03.375656 11648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:57:03.375818 11648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "af7a1665f46d4388835b89de2b3975cb"
format_stamp: "Formatted at 2026-08-09 04:57:03 on dist-test-slave-21b9"
server_key: "5f6bf1882b5fb59b3a3350b80a85213d"
server_key_iv: "53541fee15d246fc8f3a742f327dbd34"
server_key_version: "encryptionkey@0"
I20260809 04:57:03.376019 11648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:03.412530 11648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:03.416971 11648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:03.417240 11648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:03.418222 11648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:03.419649 11648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:03.419735 11648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:03.419867 11648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:03.419929 11648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:03.451520 11648 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:43019
I20260809 04:57:03.451562 11842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:43019 every 8 connection(s)
I20260809 04:57:03.452838 11648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:57:03.455587 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 11648
I20260809 04:57:03.455811 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:57:03.456635 30671 external_mini_cluster.cc:1428] Setting key 7541dba201759fb110197a9220af0b17
Aug 09 04:57:03 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:57:03.474860 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:03.455473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:56409 (local address 127.29.243.254:35297)
0809 04:57:03.455932 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:03.455936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:03.457188 (+ 1252us) server_negotiation.cc:407] Connection header received
0809 04:57:03.459435 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:03.459439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:03.459503 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:03.459594 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:03.461575 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:03.462298 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:03.464332 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:03.464785 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:03.464883 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:03.468865 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:03.468870 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:03.468874 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:03.468907 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:03.471671 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:03.472390 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:03.472396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:03.472400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:03.472557 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:03.473070 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:03.473075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:03.473077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:03.473421 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:03.473564 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:03.474302 (+ 738us) server_negotiation.cc:299] Negotiation successful
0809 04:57:03.474533 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":229,"threads_started":1}
I20260809 04:57:03.475785 11847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:03.456210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:56409)
0809 04:57:03.457025 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:03.457072 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:03.459106 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:03.459881 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:03.459894 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:03.460608 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:03.461417 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:03.461443 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:03.462446 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:03.462452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:03.464198 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:03.464213 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:03.465009 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:03.465016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:03.465277 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:03.465999 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:03.466034 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:03.468624 (+ 2590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:03.471811 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:03.471818 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:03.471831 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:03.472255 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:03.472743 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:03.472747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:03.472750 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:03.472949 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:03.473576 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:03.473582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:03.473910 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 04:57:03.474147 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 04:57:03.474551 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":316,"threads_started":1}
I20260809 04:57:03.477421 11843 heartbeater.cc:347] Connected to a master server at 127.29.243.254:35297
I20260809 04:57:03.477757 11843 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:03.478688 11843 heartbeater.cc:511] Master 127.29.243.254:35297 requested a full tablet report, sending...
I20260809 04:57:03.481101 11555 ts_manager.cc:194] Registered new tserver with Master: af7a1665f46d4388835b89de2b3975cb (127.29.243.193:43019)
I20260809 04:57:03.482641 11555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:56409
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:57:03.519683 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35297
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:57:03.870260 11850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:03.870575 11850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:03.870734 11850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:03.878332 11850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:03.878472 11850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:03.878643 11850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:57:03.888779 11850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:03.890149 11850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:03.891608 11850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:03.905062 11857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:03.905189 11861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:03.905318 11856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:03.905797 11850 server_base.cc:1034] running on GCE node
I20260809 04:57:03.906394 11850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:03.907292 11850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:03.908507 11850 hybrid_clock.cc:648] HybridClock initialized: now 1786251423908450 us; error 48 us; skew 500 ppm
I20260809 04:57:03.908803 11850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:03 dist-test-slave-21b9 krb5kdc[11486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:03.911854 11850 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:57:03.913295 11850 webserver.cc:460] Webserver started at http://127.29.243.194:44701/ using document root <none> and password file <none>
I20260809 04:57:03.913959 11850 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:03.914081 11850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:03.914350 11850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:03.917137 11850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "277a2e8276264c2189e3281584c7d738"
format_stamp: "Formatted at 2026-08-09 04:57:03 on dist-test-slave-21b9"
server_key: "aba1519052e8a4db4983b4e766246997"
server_key_iv: "fc7ef50c18d35f339360c3d6a90afa73"
server_key_version: "encryptionkey@0"
I20260809 04:57:03.917860 11850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "277a2e8276264c2189e3281584c7d738"
format_stamp: "Formatted at 2026-08-09 04:57:03 on dist-test-slave-21b9"
server_key: "aba1519052e8a4db4983b4e766246997"
server_key_iv: "fc7ef50c18d35f339360c3d6a90afa73"
server_key_version: "encryptionkey@0"
I20260809 04:57:03.922917 11850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 04:57:03.926520 11875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:03.927992 11850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:57:03.928154 11850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "277a2e8276264c2189e3281584c7d738"
format_stamp: "Formatted at 2026-08-09 04:57:03 on dist-test-slave-21b9"
server_key: "aba1519052e8a4db4983b4e766246997"
server_key_iv: "fc7ef50c18d35f339360c3d6a90afa73"
server_key_version: "encryptionkey@0"
I20260809 04:57:03.928336 11850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:03.949476 11850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:03.953547 11850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:03.953810 11850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:03.954794 11850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:03.956193 11850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:03.956284 11850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:03.956403 11850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:03.956463 11850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:03.987635 11850 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:40773
I20260809 04:57:03.987700 12041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:40773 every 8 connection(s)
I20260809 04:57:03.988896 11850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:57:03.995653 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 11850
I20260809 04:57:03.995857 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:57:03.996409 30671 external_mini_cluster.cc:1428] Setting key 818b7bba78c28ef163a99ecd4c0e43bd
Aug 09 04:57:03 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:57:04.011045 12044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:03.991529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:51117 (local address 127.29.243.254:35297)
0809 04:57:03.991949 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:03.991957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:03.992931 (+ 974us) server_negotiation.cc:407] Connection header received
0809 04:57:03.994729 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:03.994734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:03.994818 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:03.994916 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:03.997020 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:03.997844 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:03.999942 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.000373 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.000474 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:04.004396 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:04.004402 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:04.004407 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:04.004447 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:04.007406 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:04.008176 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:04.008182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:04.008185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:04.008332 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:04.008834 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:04.008838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:04.008841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:04.009186 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:04.009316 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:04.010441 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0809 04:57:04.010671 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":204,"threads_started":1}
I20260809 04:57:04.012130 12047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:03.992044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:51117)
0809 04:57:03.992754 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:03.992804 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:03.994456 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:03.995159 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:03.995167 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:03.995980 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:03.996781 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:03.996815 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:03.998015 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:03.998021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:03.999787 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:03.999802 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.000601 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.000609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:04.000882 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:04.001646 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:04.001675 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:04.004124 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:04.007532 (+ 3408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:04.007541 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:04.007557 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:04.008051 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:04.008465 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:04.008470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:04.008473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:04.008706 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:04.009389 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:04.009397 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:04.009812 (+ 415us) client_negotiation.cc:769] Sending connection context
0809 04:57:04.010111 (+ 299us) client_negotiation.cc:240] Negotiation successful
0809 04:57:04.010719 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":269,"threads_started":1}
I20260809 04:57:04.014513 12042 heartbeater.cc:347] Connected to a master server at 127.29.243.254:35297
I20260809 04:57:04.014854 12042 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:04.015770 12042 heartbeater.cc:511] Master 127.29.243.254:35297 requested a full tablet report, sending...
I20260809 04:57:04.017566 11555 ts_manager.cc:194] Registered new tserver with Master: 277a2e8276264c2189e3281584c7d738 (127.29.243.194:40773)
I20260809 04:57:04.018384 11555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:51117
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:57:04.056489 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35297
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:57:04.389147 12051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:04.389447 12051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:04.389587 12051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:04.397203 12051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:04.397328 12051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:04.397487 12051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:57:04.407019 12051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:35297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:04.408344 12051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:04.409714 12051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:04.422972 12069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:04.422919 12065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:04.422946 12066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:04.423979 12051 server_base.cc:1034] running on GCE node
I20260809 04:57:04.424556 12051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:04.425465 12051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:04.426666 12051 hybrid_clock.cc:648] HybridClock initialized: now 1786251424426608 us; error 50 us; skew 500 ppm
I20260809 04:57:04.426949 12051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:04 dist-test-slave-21b9 krb5kdc[11486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251424, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:04.430022 12051 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:57:04.431504 12051 webserver.cc:460] Webserver started at http://127.29.243.195:39017/ using document root <none> and password file <none>
I20260809 04:57:04.432217 12051 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:04.432322 12051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:04.432662 12051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:04.435596 12051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "7b66c16c262d4abe89b7615048db04dc"
format_stamp: "Formatted at 2026-08-09 04:57:04 on dist-test-slave-21b9"
server_key: "c91696473a50f5fbe282dfcfb61db9d2"
server_key_iv: "7fceaf829e2b74cdb4fea58753666b1a"
server_key_version: "encryptionkey@0"
I20260809 04:57:04.436367 12051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "7b66c16c262d4abe89b7615048db04dc"
format_stamp: "Formatted at 2026-08-09 04:57:04 on dist-test-slave-21b9"
server_key: "c91696473a50f5fbe282dfcfb61db9d2"
server_key_iv: "7fceaf829e2b74cdb4fea58753666b1a"
server_key_version: "encryptionkey@0"
I20260809 04:57:04.441627 12051 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:57:04.445077 12079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:04.446425 12051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:57:04.446583 12051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "7b66c16c262d4abe89b7615048db04dc"
format_stamp: "Formatted at 2026-08-09 04:57:04 on dist-test-slave-21b9"
server_key: "c91696473a50f5fbe282dfcfb61db9d2"
server_key_iv: "7fceaf829e2b74cdb4fea58753666b1a"
server_key_version: "encryptionkey@0"
I20260809 04:57:04.446764 12051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:04.462065 12051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:04.466373 12051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:04.466629 12051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:04.467585 12051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:04.468909 12051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:04.468983 12051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:04.469118 12051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:04.469169 12051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:04.486274 11843 heartbeater.cc:503] Master 127.29.243.254:35297 was elected leader, sending a full tablet report...
I20260809 04:57:04.500207 12051 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:35343
I20260809 04:57:04.500216 12258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:35343 every 8 connection(s)
I20260809 04:57:04.501521 12051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:57:04.510704 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 12051
I20260809 04:57:04.510886 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:57:04.511380 30671 external_mini_cluster.cc:1428] Setting key e33cbc6d107adfd1c8a8f5e59c3793f8
Aug 09 04:57:04 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251424, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:57:04.521158 12044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:04.504211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:44381 (local address 127.29.243.254:35297)
0809 04:57:04.504392 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:04.504397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:04.505520 (+ 1123us) server_negotiation.cc:407] Connection header received
0809 04:57:04.507311 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:04.507316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:04.507411 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:04.507499 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:04.508897 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.509639 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.511249 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.511866 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.512026 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:04.516072 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:04.516077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:04.516081 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:04.516114 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:04.518549 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:04.519208 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:04.519211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:04.519214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:04.519373 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:04.519732 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:04.519736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:04.519738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:04.520013 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:04.520123 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:04.520741 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 04:57:04.520926 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:57:04.522269 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:04.504637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:44381)
0809 04:57:04.505370 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:04.505409 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:04.507030 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:04.507695 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:04.507702 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:04.508206 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:04.508745 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:04.508763 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.509775 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.509779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:04.511103 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:04.511116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.512156 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.512164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:04.512408 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:04.513169 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:04.513192 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:04.515859 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:04.518676 (+ 2817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:04.518683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:04.518697 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:04.519116 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:04.519465 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:04.519469 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:04.519471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:04.519651 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:04.520119 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:04.520125 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:04.520426 (+ 301us) client_negotiation.cc:769] Sending connection context
0809 04:57:04.520655 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 04:57:04.521035 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":290,"threads_started":1}
I20260809 04:57:04.523779 12259 heartbeater.cc:347] Connected to a master server at 127.29.243.254:35297
I20260809 04:57:04.524077 12259 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:04.524821 12259 heartbeater.cc:511] Master 127.29.243.254:35297 requested a full tablet report, sending...
I20260809 04:57:04.526347 11555 ts_manager.cc:194] Registered new tserver with Master: 7b66c16c262d4abe89b7615048db04dc (127.29.243.195:35343)
I20260809 04:57:04.527096 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:57:04.527235 11555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:44381
WARNING: no policy specified for oryx/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab.
Entry for principal oryx/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab.
WARNING: no policy specified for oryx/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab.
Entry for principal oryx/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab.
WARNING: no policy specified for oryx/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab.
Entry for principal oryx/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab.
I20260809 04:57:04.684170 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 11648
Aug 09 04:57:04 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:57:04.699790 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:04.685405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.29.243.193:43019)
0809 04:57:04.686135 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:04.686143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:04.686171 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:57:04.686239 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:04.686246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:04.686477 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:04.686615 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:04.687998 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.689019 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.690221 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.690798 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.690895 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:04.694707 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:04.694719 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:04.694734 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:04.694766 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:04.697430 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:04.698023 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:04.698030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:04.698033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:04.698156 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:04.698436 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:04.698439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:04.698441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:04.698769 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:04.699016 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:04.699189 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 04:57:04.699424 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":374,"threads_started":1}
I20260809 04:57:04.701098 11836 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:04.785167 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 11648
I20260809 04:57:04.811190 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 11850
Aug 09 04:57:04 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:57:04.829077 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:04.812184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33570 (local address 127.29.243.194:40773)
0809 04:57:04.812778 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:04.812786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:04.812814 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:57:04.812878 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:04.812885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:04.813102 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:04.813231 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:04.814559 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.815781 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.817699 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.818295 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.818402 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:04.823249 (+ 4847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:04.823262 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:04.823354 (+ 92us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:04.823404 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:04.826240 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:04.826901 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:04.826908 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:04.826911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:04.827045 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:04.827422 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:04.827427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:04.827430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:04.827810 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:04.828106 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:04.828444 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 04:57:04.828726 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":300,"threads_started":1}
I20260809 04:57:04.830415 12036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:04.911408 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 11850
I20260809 04:57:04.934333 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 12051
Aug 09 04:57:04 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:57:04.951624 12284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:04.935420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.29.243.195:35343)
0809 04:57:04.936064 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:04.936071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:04.936103 (+ 32us) server_negotiation.cc:407] Connection header received
0809 04:57:04.936165 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:04.936172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:04.936390 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:04.936524 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:04.937924 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.939030 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.940583 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:04.941179 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:04.941284 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:04.945797 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:04.945809 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:04.945828 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:04.945866 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:04.948671 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:04.949356 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:04.949363 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:04.949366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:04.949517 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:04.949894 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:04.949897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:04.949899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:04.950290 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:04.950621 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:04.950954 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 04:57:04.951224 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":300,"threads_started":1}
I20260809 04:57:04.952932 12254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:05.034148 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 12051
I20260809 04:57:05.059914 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 11505
I20260809 04:57:05.061244 11601 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:05.165793 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 11505
I20260809 04:57:05.191838 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35297
--webserver_interface=127.29.243.254
--webserver_port=38015
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:35297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:57:05.533730 12287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:05.534039 12287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:05.534180 12287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:05.541447 12287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:57:05.541558 12287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:05.541633 12287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:05.541715 12287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:57:05.541774 12287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:57:05.551541 12287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:35297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=38015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:05.552877 12287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:05.554209 12287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:05.566085 12294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:05.566089 12296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:05.566658 12298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:05.567127 12287 server_base.cc:1034] running on GCE node
I20260809 04:57:05.567925 12287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:05.569290 12287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:05.570489 12287 hybrid_clock.cc:648] HybridClock initialized: now 1786251425570438 us; error 47 us; skew 500 ppm
I20260809 04:57:05.570796 12287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:05 dist-test-slave-21b9 krb5kdc[11486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251425, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:05.573971 12287 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:57:05.575526 12287 webserver.cc:460] Webserver started at http://127.29.243.254:38015/ using document root <none> and password file <none>
I20260809 04:57:05.576238 12287 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:05.576360 12287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:05.581884 12287 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:57:05.584805 12310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:05.586155 12287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:57:05.586318 12287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "0ec96aba9263448aad6314c8533e8026"
format_stamp: "Formatted at 2026-08-09 04:57:02 on dist-test-slave-21b9"
server_key: "9759fd7e03e8c2bb604dcbe9c86e0a37"
server_key_iv: "d73bb2962596a7ad2981d5c3f9fb4f15"
server_key_version: "encryptionkey@0"
I20260809 04:57:05.587311 12287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:05.599253 12287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:05.603391 12287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:05.603633 12287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:05.629896 12287 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:35297
I20260809 04:57:05.629915 12387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:35297 every 8 connection(s)
I20260809 04:57:05.631526 12287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:57:05.638370 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 12287
I20260809 04:57:05.638823 12389 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:57:05.639381 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:43019
--local_ip_for_outbound_sockets=127.29.243.193
--tserver_master_addrs=127.29.243.254:35297
--webserver_port=36763
--webserver_interface=127.29.243.193
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.243.193
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:57:05.643234 12389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026: Bootstrap starting.
I20260809 04:57:05.665860 12389 log.cc:824] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026: Log is configured to *not* fsync() on all Append() calls
I20260809 04:57:05.677371 12389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:57:05.677994 12389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026: Bootstrap complete.
I20260809 04:57:05.691169 12389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } }
I20260809 04:57:05.692431 12389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec96aba9263448aad6314c8533e8026, State: Initialized, Role: FOLLOWER
I20260809 04:57:05.693186 12389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } }
I20260809 04:57:05.693485 12389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:57:05.693625 12389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:57:05.693812 12389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:57:05.699505 12389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } }
I20260809 04:57:05.700111 12389 leader_election.cc:302] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ec96aba9263448aad6314c8533e8026; no voters:
I20260809 04:57:05.700758 12389 leader_election.cc:288] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:57:05.700898 12393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:57:05.702050 12393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [term 2 LEADER]: Becoming Leader. State: Replica: 0ec96aba9263448aad6314c8533e8026, State: Running, Role: LEADER
I20260809 04:57:05.702569 12393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } }
I20260809 04:57:05.702965 12389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:57:05.705080 12394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0ec96aba9263448aad6314c8533e8026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } } }
I20260809 04:57:05.705056 12395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ec96aba9263448aad6314c8533e8026. Latest consensus state: current_term: 2 leader_uuid: "0ec96aba9263448aad6314c8533e8026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec96aba9263448aad6314c8533e8026" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35297 } } }
I20260809 04:57:05.705479 12394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [sys.catalog]: This master's current role is: LEADER
I20260809 04:57:05.705555 12395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026 [sys.catalog]: This master's current role is: LEADER
I20260809 04:57:05.717013 12401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:57:05.723214 12401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:57:05.724640 12401 catalog_manager.cc:1259] Loaded cluster ID: 40b155a017004377995a9f7fd576fc0e
I20260809 04:57:05.724745 12401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:57:05.728644 12401 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:57:05.730166 12401 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0ec96aba9263448aad6314c8533e8026: Loaded TSK: 0
I20260809 04:57:05.731828 12401 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:57:06.007149 12391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:06.007486 12391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:06.007629 12391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:06.018388 12391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:06.018513 12391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:06.018697 12391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:57:06.029022 12391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.193.keytab
--principal=oryx/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:43019
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=36763
--tserver_master_addrs=127.29.243.254:35297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:06.030391 12391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:06.031787 12391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:06.046526 12422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:06.046746 12424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:06.047578 12426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:06.048113 12391 server_base.cc:1034] running on GCE node
I20260809 04:57:06.048784 12391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:06.049814 12391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:06.051038 12391 hybrid_clock.cc:648] HybridClock initialized: now 1786251426050973 us; error 64 us; skew 500 ppm
I20260809 04:57:06.051414 12391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251426, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:06.054590 12391 init.cc:377] Logged in from keytab as oryx/127.29.243.193@KRBTEST.COM (short username oryx)
I20260809 04:57:06.056197 12391 webserver.cc:460] Webserver started at http://127.29.243.193:36763/ using document root <none> and password file <none>
I20260809 04:57:06.056929 12391 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:06.057057 12391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:06.062614 12391 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 04:57:06.065435 12437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:06.066836 12391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:57:06.066999 12391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "af7a1665f46d4388835b89de2b3975cb"
format_stamp: "Formatted at 2026-08-09 04:57:03 on dist-test-slave-21b9"
server_key: "5f6bf1882b5fb59b3a3350b80a85213d"
server_key_iv: "53541fee15d246fc8f3a742f327dbd34"
server_key_version: "encryptionkey@0"
I20260809 04:57:06.067999 12391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:06.085688 12391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:06.090072 12391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:06.090327 12391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:06.091409 12391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:06.092902 12391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:06.093004 12391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:06.093111 12391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:06.093190 12391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:06.124248 12391 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:43019
I20260809 04:57:06.124271 12615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:43019 every 8 connection(s)
I20260809 04:57:06.125764 12391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:57:06.129381 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 12391
I20260809 04:57:06.130054 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:40773
--local_ip_for_outbound_sockets=127.29.243.194
--tserver_master_addrs=127.29.243.254:35297
--webserver_port=44701
--webserver_interface=127.29.243.194
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.243.194
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:06.145942 12624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:06.128853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:60915)
0809 04:57:06.130822 (+ 1969us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:06.130881 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:06.133107 (+ 2226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:06.134526 (+ 1419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:06.134536 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:06.135229 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:06.136006 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.136032 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.137668 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.137672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.138896 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.138909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.140173 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.140210 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.140518 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.141195 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:06.141225 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:06.145030 (+ 3805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1673,"thread_start_us":1436,"threads_started":1}
W20260809 04:57:06.146556 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:06.128827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:60915 (local address 127.29.243.254:35297)
0809 04:57:06.129673 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:06.129689 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:06.131025 (+ 1336us) server_negotiation.cc:407] Connection header received
0809 04:57:06.133671 (+ 2646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:06.133684 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:06.134025 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:06.134305 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:06.136219 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.137501 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.139046 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.139851 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.140040 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.145466 (+ 5426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:60915: BlockingRecv error: recv got EOF from 127.29.243.193:60915 (error 108)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":276,"threads_started":1}
W20260809 04:57:06.146720 12616 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:35297 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.243.254:35297: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:06.158124 12624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:06.147300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:48183)
0809 04:57:06.147470 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:06.147496 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:06.147593 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:06.147872 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:06.147877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:06.148196 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:06.148330 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.148338 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.151349 (+ 3011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.151356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.152922 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.152938 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.153853 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.153861 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.154090 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.154710 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:06.154733 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:06.157859 (+ 3126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 04:57:06.158279 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:06.147450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:48183 (local address 127.29.243.254:35297)
0809 04:57:06.147619 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:06.147624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:06.147642 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:57:06.147690 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:06.147696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:06.147756 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:06.147834 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:06.148463 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.149445 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.153073 (+ 3628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.153668 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.153785 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.158074 (+ 4289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:48183: BlockingRecv error: recv got EOF from 127.29.243.193:48183 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:06.166438 12624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:06.159013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:48269)
0809 04:57:06.159209 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:06.159224 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:06.159334 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:06.159872 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:06.159877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:06.160158 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:06.160293 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.160301 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.161274 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.161278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.162318 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.162330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.162973 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.162979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.163187 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.163726 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:06.163742 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:06.166164 (+ 2422us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 04:57:06.166621 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:06.159466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:48269 (local address 127.29.243.254:35297)
0809 04:57:06.159625 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:06.159629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:06.159643 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:57:06.159680 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:06.159683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:06.159746 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:06.159825 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:06.160424 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.161142 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.162450 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.162847 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.162970 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.166372 (+ 3402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:48269: BlockingRecv error: recv got EOF from 127.29.243.193:48269 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 04:57:06.166821 12616 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:57:06.479131 12625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:06.479414 12625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:06.479517 12625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:06.486470 12625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:06.486565 12625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:06.486680 12625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:57:06.496126 12625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.194.keytab
--principal=oryx/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:40773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=44701
--tserver_master_addrs=127.29.243.254:35297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:06.497339 12625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:06.498608 12625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:06.511435 12631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:06.511381 12640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:06.511516 12636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:06.512308 12625 server_base.cc:1034] running on GCE node
I20260809 04:57:06.512897 12625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:06.513811 12625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:06.515010 12625 hybrid_clock.cc:648] HybridClock initialized: now 1786251426514958 us; error 46 us; skew 500 ppm
I20260809 04:57:06.515331 12625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251426, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:06.518381 12625 init.cc:377] Logged in from keytab as oryx/127.29.243.194@KRBTEST.COM (short username oryx)
I20260809 04:57:06.519867 12625 webserver.cc:460] Webserver started at http://127.29.243.194:44701/ using document root <none> and password file <none>
I20260809 04:57:06.520533 12625 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:06.520663 12625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:06.525895 12625 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 04:57:06.528563 12648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:06.529870 12625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:57:06.530030 12625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "277a2e8276264c2189e3281584c7d738"
format_stamp: "Formatted at 2026-08-09 04:57:03 on dist-test-slave-21b9"
server_key: "aba1519052e8a4db4983b4e766246997"
server_key_iv: "fc7ef50c18d35f339360c3d6a90afa73"
server_key_version: "encryptionkey@0"
I20260809 04:57:06.530961 12625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:06.543820 12625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:06.547451 12625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:06.547706 12625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:06.548630 12625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:06.549990 12625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:06.550099 12625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:06.550202 12625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:06.550272 12625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:06.580462 12625 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:40773
I20260809 04:57:06.580528 12825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:40773 every 8 connection(s)
I20260809 04:57:06.581918 12625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:57:06.590623 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 12625
I20260809 04:57:06.591208 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:35343
--local_ip_for_outbound_sockets=127.29.243.195
--tserver_master_addrs=127.29.243.254:35297
--webserver_port=39017
--webserver_interface=127.29.243.195
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.243.195
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:57:06.597491 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:06.584437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:49729 (local address 127.29.243.254:35297)
0809 04:57:06.584633 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:06.584637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:06.585777 (+ 1140us) server_negotiation.cc:407] Connection header received
0809 04:57:06.587645 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:06.587651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:06.587735 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:06.587824 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:06.589339 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.590117 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.591989 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.592406 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.592530 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.597258 (+ 4728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:49729: BlockingRecv error: recv got EOF from 127.29.243.194:49729 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 04:57:06.597997 12829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:06.584886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:49729)
0809 04:57:06.585611 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:06.585657 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:06.587374 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:06.588078 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:06.588087 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:06.588595 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:06.589170 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.589188 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.590276 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.590280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.591863 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.591879 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.592627 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.592635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.592858 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.593512 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:06.593535 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:06.597015 (+ 3480us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":279,"threads_started":1}
W20260809 04:57:06.598619 12827 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:35297 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.243.254:35297: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:06.606293 12829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:06.599175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:59681)
0809 04:57:06.599346 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:06.599364 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:06.599454 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:06.599794 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:06.599798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:06.600109 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:06.600237 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.600245 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.601194 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.601200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.602071 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.602082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.602739 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.602745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.602904 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.603468 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:06.603485 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:06.606019 (+ 2534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 04:57:06.606479 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:06.599357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:59681 (local address 127.29.243.254:35297)
0809 04:57:06.599554 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:06.599560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:06.599576 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:57:06.599618 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:06.599622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:06.599676 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:06.599754 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:06.600372 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.601064 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.602204 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.602604 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.602710 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.606253 (+ 3543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:59681: BlockingRecv error: recv got EOF from 127.29.243.194:59681 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:06.614547 12829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:06.607178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:43311)
0809 04:57:06.607352 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:06.607368 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:06.607446 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:06.607831 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:06.607836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:06.608109 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:06.608245 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.608253 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.609186 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.609190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.610080 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:06.610091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.610738 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.610743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:06.610924 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.611492 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:06.611514 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:06.614262 (+ 2748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 04:57:06.614809 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:06.607398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:43311 (local address 127.29.243.254:35297)
0809 04:57:06.607603 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:06.607607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:06.607620 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:57:06.607657 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:06.607660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:06.607712 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:06.607788 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:06.608377 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.609043 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.610210 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:06.610607 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:06.610712 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:06.614491 (+ 3779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:43311: BlockingRecv error: recv got EOF from 127.29.243.194:43311 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 04:57:06.615036 12827 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:57:06.931859 12830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:06.932155 12830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:06.932258 12830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:06.939543 12830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:06.939661 12830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:06.939779 12830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:57:06.949631 12830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/krb5kdc/oryx_127.29.243.195.keytab
--principal=oryx/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:35343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=39017
--tserver_master_addrs=127.29.243.254:35297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:06.950961 12830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:06.952322 12830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:06.965018 12835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:06.965058 12837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:06.965250 12839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:06.965950 12830 server_base.cc:1034] running on GCE node
I20260809 04:57:06.966538 12830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:06.967526 12830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:06.968729 12830 hybrid_clock.cc:648] HybridClock initialized: now 1786251426968681 us; error 44 us; skew 500 ppm
I20260809 04:57:06.969043 12830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:06 dist-test-slave-21b9 krb5kdc[11486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251426, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:06.972273 12830 init.cc:377] Logged in from keytab as oryx/127.29.243.195@KRBTEST.COM (short username oryx)
I20260809 04:57:06.973770 12830 webserver.cc:460] Webserver started at http://127.29.243.195:39017/ using document root <none> and password file <none>
I20260809 04:57:06.974453 12830 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:06.974575 12830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:06.979956 12830 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260809 04:57:06.982690 12847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:06.984091 12830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 04:57:06.984239 12830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "7b66c16c262d4abe89b7615048db04dc"
format_stamp: "Formatted at 2026-08-09 04:57:04 on dist-test-slave-21b9"
server_key: "c91696473a50f5fbe282dfcfb61db9d2"
server_key_iv: "7fceaf829e2b74cdb4fea58753666b1a"
server_key_version: "encryptionkey@0"
I20260809 04:57:06.985183 12830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:07.001956 12830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:07.005841 12830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:07.006098 12830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:07.007093 12830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:07.008975 12830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:07.009089 12830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:07.009196 12830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:07.009268 12830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:07.040055 12830 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:35343
I20260809 04:57:07.040127 13023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:35343 every 8 connection(s)
I20260809 04:57:07.041635 12830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:57:07.048892 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 12830
Aug 09 04:57:07 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:07 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:57:07.058252 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:07.044114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:51903 (local address 127.29.243.254:35297)
0809 04:57:07.044304 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:07.044311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:07.045496 (+ 1185us) server_negotiation.cc:407] Connection header received
0809 04:57:07.047351 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:07.047356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:07.047439 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:07.047529 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:07.049426 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.050494 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.052371 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.052902 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.053001 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:07.057953 (+ 4952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:51903: BlockingRecv error: recv got EOF from 127.29.243.195:51903 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":62}
I20260809 04:57:07.058732 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:07.044624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:51903)
0809 04:57:07.045333 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:07.045373 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:07.047025 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:07.047780 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:07.047788 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:07.048321 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:07.049188 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:07.049216 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.050721 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.050726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:07.052215 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:07.052228 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.053129 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.053137 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:07.053408 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:07.054152 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:07.054176 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:07.057692 (+ 3516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":274,"threads_started":1}
W20260809 04:57:07.059496 13025 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:35297 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.243.254:35297: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Aug 09 04:57:07 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:07.066206 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:07.050045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58496 (local address 127.29.243.254:35297)
0809 04:57:07.050835 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:07.050841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:07.050857 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:57:07.050903 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:07.050908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:07.050982 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:07.051091 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:07.052547 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.053572 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.054933 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.055497 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.055608 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:07.058888 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:07.058899 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:07.058921 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:07.058971 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:07.062769 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:07.063471 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:07.063479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:07.063486 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:07.063688 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:07.064088 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:07.064092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:07.064095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:07.064742 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:07.065077 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:07.065451 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 04:57:07.065849 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":677,"spinlock_wait_cycles":7936,"thread_start_us":543,"threads_started":1}
Aug 09 04:57:07 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:07.068084 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:07.060107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:40211)
0809 04:57:07.060258 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:07.060274 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:07.060351 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:07.060594 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:07.060597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:07.060876 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:07.061008 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:07.061015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.062040 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.062055 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:07.063233 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:07.063246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.064083 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.064089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:07.064262 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:07.064770 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:07.064792 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:07.067818 (+ 3026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 04:57:07.068171 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:07.060209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:40211 (local address 127.29.243.254:35297)
0809 04:57:07.060360 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:07.060364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:07.060378 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:57:07.060449 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:07.060452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:07.060504 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:07.060569 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:07.061139 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.061897 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.063380 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.063926 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.064064 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:07.067970 (+ 3906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:40211: BlockingRecv error: recv got EOF from 127.29.243.195:40211 (error 108)
Metrics: {"server-negotiator.queue_time_us":42,"spinlock_wait_cycles":4864}
Aug 09 04:57:07 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:07 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:07.075780 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:07.068766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:39709)
0809 04:57:07.068996 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:07.069015 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:07.069114 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:07.069382 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:07.069385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:07.069695 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:07.069878 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:07.069890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.070982 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.070986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:07.072004 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:07.072016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.072605 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.072609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:07.072761 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:07.073177 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:07.073193 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:07.075573 (+ 2380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129}
W20260809 04:57:07.076005 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:07.068843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:39709 (local address 127.29.243.254:35297)
0809 04:57:07.069099 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:07.069105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:07.069123 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:57:07.069197 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:07.069202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:07.069260 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:07.069336 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:07.070026 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.070857 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.072121 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.072504 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.072594 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:07.075763 (+ 3169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:39709: BlockingRecv error: recv got EOF from 127.29.243.195:39709 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
W20260809 04:57:07.076097 13025 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 04:57:07 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:07 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:07.176156 13031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:07.167566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:35401)
0809 04:57:07.168123 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:07.168140 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:07.168282 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:07.168685 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:07.168690 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:07.169199 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:07.169401 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:07.169411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.170406 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.170412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:07.171491 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:07.171505 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.172340 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.172347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:07.172514 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:07.173054 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:07.173071 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:07.175825 (+ 2754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":302,"threads_started":1}
W20260809 04:57:07.176301 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:07.167648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:35401 (local address 127.29.243.254:35297)
0809 04:57:07.167887 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:07.167900 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:07.168224 (+ 324us) server_negotiation.cc:407] Connection header received
0809 04:57:07.168443 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:07.168447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:07.168523 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:07.168603 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:07.169547 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.170256 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.171630 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.172145 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.172277 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:07.176059 (+ 3782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:35401: BlockingRecv error: recv got EOF from 127.29.243.193:35401 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 04:57:07 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:07 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:07.624241 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:07.615648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:45801)
0809 04:57:07.616126 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:07.616143 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:07.616270 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:07.616634 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:07.616640 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:07.617256 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:07.617530 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:07.617546 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.618790 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.618794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:07.619817 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:07.619830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.620576 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.620580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:07.620697 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:07.621268 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:07.621286 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:07.623989 (+ 2703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":238,"threads_started":1}
W20260809 04:57:07.624459 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:07.615723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:45801 (local address 127.29.243.254:35297)
0809 04:57:07.615947 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:07.615953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:07.616234 (+ 281us) server_negotiation.cc:407] Connection header received
0809 04:57:07.616385 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:07.616390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:07.616477 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:07.616563 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:07.617696 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.618643 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.619972 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:07.620425 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:07.620548 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:07.624219 (+ 3671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:45801: BlockingRecv error: recv got EOF from 127.29.243.194:45801 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 04:57:08 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:08 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:08.088722 13033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:08.076798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:49787)
0809 04:57:08.077308 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:08.077325 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:08.077462 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:08.077813 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:08.077817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:08.078291 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:08.078494 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:08.078503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.079580 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.079586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:08.080526 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:08.080537 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.081232 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.081239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:08.081391 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:08.085691 (+ 4300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:08.085709 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:08.088372 (+ 2663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":264,"threads_started":1}
W20260809 04:57:08.088867 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:08.076926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:49787 (local address 127.29.243.254:35297)
0809 04:57:08.077170 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:08.077176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:08.077408 (+ 232us) server_negotiation.cc:407] Connection header received
0809 04:57:08.077590 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:08.077595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:08.077665 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:08.077747 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:08.078637 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.079439 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.080663 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.081060 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.081167 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:08.088608 (+ 7441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:49787: BlockingRecv error: recv got EOF from 127.29.243.195:49787 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 09 04:57:08 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:08 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:08.185807 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:08.177339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:46071)
0809 04:57:08.177891 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:08.177911 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:08.178088 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:08.178441 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:08.178446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:08.178956 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:08.179178 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:08.179193 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.180634 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.180639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:08.181545 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:08.181555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.182172 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.182176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:08.182288 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:08.182810 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:08.182828 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:08.185557 (+ 2729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":297,"threads_started":1}
W20260809 04:57:08.185927 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:08.177397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:46071 (local address 127.29.243.254:35297)
0809 04:57:08.177651 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:08.177657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:08.178003 (+ 346us) server_negotiation.cc:407] Connection header received
0809 04:57:08.178205 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:08.178211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:08.178290 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:08.178376 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:08.179388 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.180503 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.181653 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.182060 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.182148 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:08.185719 (+ 3571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:46071: BlockingRecv error: recv got EOF from 127.29.243.193:46071 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 04:57:08 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:08 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:08.634229 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:08.625177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:39575)
0809 04:57:08.625716 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:08.625732 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:08.625869 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:08.626200 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:08.626205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:08.626663 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:08.626891 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:08.626905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.628232 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.628236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:08.629266 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:08.629280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.630188 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.630193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:08.630306 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:08.630907 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:08.630923 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:08.633974 (+ 3051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":298,"threads_started":1}
W20260809 04:57:08.634464 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:08.625279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:39575 (local address 127.29.243.254:35297)
0809 04:57:08.625480 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:08.625487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:08.625817 (+ 330us) server_negotiation.cc:407] Connection header received
0809 04:57:08.625998 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:08.626003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:08.626069 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:08.626151 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:08.627061 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.628113 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.629399 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:08.630047 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:08.630168 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:08.634212 (+ 4044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:39575: BlockingRecv error: recv got EOF from 127.29.243.194:39575 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 04:57:09 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:09 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:09.097937 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:09.089810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:37741)
0809 04:57:09.090329 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:09.090346 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:09.090470 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:09.090784 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:09.090788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:09.091249 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:09.091465 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:09.091476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.092570 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.092575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:09.093457 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:09.093470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.094142 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.094147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:09.094257 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:09.094840 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:09.094857 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:09.097683 (+ 2826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":285,"threads_started":1}
W20260809 04:57:09.098169 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:09.090000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:37741 (local address 127.29.243.254:35297)
0809 04:57:09.090233 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:09.090240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:09.090421 (+ 181us) server_negotiation.cc:407] Connection header received
0809 04:57:09.090566 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:09.090569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:09.090670 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:09.090746 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:09.091616 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.092436 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.093584 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.094011 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.094106 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:09.097911 (+ 3805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:37741: BlockingRecv error: recv got EOF from 127.29.243.195:37741 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 04:57:09 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:09 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:09.195464 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:09.186761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:46523)
0809 04:57:09.187349 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:09.187365 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:09.187513 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:09.187838 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:09.187843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:09.188221 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:09.188426 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:09.188462 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.189807 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.189813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:09.190743 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:09.190756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.191613 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.191619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:09.191795 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:09.192404 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:09.192423 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:09.195153 (+ 2730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":292,"threads_started":1}
W20260809 04:57:09.195582 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:09.186878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:46523 (local address 127.29.243.254:35297)
0809 04:57:09.187093 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:09.187100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:09.187440 (+ 340us) server_negotiation.cc:407] Connection header received
0809 04:57:09.187638 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:09.187643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:09.187709 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:09.187788 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:09.188626 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.189657 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.190885 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.191469 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.191579 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:09.195372 (+ 3793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:46523: BlockingRecv error: recv got EOF from 127.29.243.193:46523 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 04:57:09 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:09 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:09.643463 13042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:09.635189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:38291)
0809 04:57:09.635837 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:09.635858 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:09.635989 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:09.636276 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:09.636280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:09.636616 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:09.636795 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:09.636804 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.637917 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.637922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:09.638847 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:09.638861 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.639601 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.639605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:09.639734 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:09.640311 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:09.640331 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:09.643151 (+ 2820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":334,"threads_started":1}
W20260809 04:57:09.643656 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:09.635326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:38291 (local address 127.29.243.254:35297)
0809 04:57:09.635560 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:09.635566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:09.635961 (+ 395us) server_negotiation.cc:407] Connection header received
0809 04:57:09.636088 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:09.636091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:09.636158 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:09.636254 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:09.636932 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.637751 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.638994 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:09.639449 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:09.639542 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:09.643401 (+ 3859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:38291: BlockingRecv error: recv got EOF from 127.29.243.194:38291 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 09 04:57:10 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:10 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:10.107779 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:10.098935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:52305)
0809 04:57:10.099521 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:10.099541 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:10.099737 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:10.100280 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:10.100285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:10.100714 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:10.100903 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:10.100914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.101962 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.101968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:10.102910 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:10.102924 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.103705 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.103712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:10.103902 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:10.104518 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:10.104539 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:10.107512 (+ 2973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":309,"threads_started":1}
W20260809 04:57:10.107950 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:10.099075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:52305 (local address 127.29.243.254:35297)
0809 04:57:10.099293 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:10.099300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:10.099622 (+ 322us) server_negotiation.cc:407] Connection header received
0809 04:57:10.099933 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:10.099940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:10.100109 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:10.100256 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:10.101076 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.101819 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.103063 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.103558 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.103706 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:10.107735 (+ 4029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:52305: BlockingRecv error: recv got EOF from 127.29.243.195:52305 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 04:57:10 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:10 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:10.204751 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:10.196425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:57037)
0809 04:57:10.197093 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:10.197113 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:10.197248 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:10.197587 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:10.197592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:10.198000 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:10.198157 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:10.198166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.199159 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.199165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:10.200178 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:10.200191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.200874 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.200879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:10.201049 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:10.201634 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:10.201652 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:10.204436 (+ 2784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":421,"threads_started":1}
W20260809 04:57:10.204905 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:10.196492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:57037 (local address 127.29.243.254:35297)
0809 04:57:10.196730 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:10.196737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:10.197185 (+ 448us) server_negotiation.cc:407] Connection header received
0809 04:57:10.197383 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:10.197388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:10.197454 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:10.197533 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:10.198301 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.199023 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.200313 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.200727 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.200835 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:10.204665 (+ 3830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:57037: BlockingRecv error: recv got EOF from 127.29.243.193:57037 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 04:57:10 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:10 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:10.652552 13045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:10.644419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:51607)
0809 04:57:10.644894 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:10.644910 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:10.645055 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:10.645369 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:10.645374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:10.645762 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:10.645916 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:10.645925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.646928 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.646933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:10.647960 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:10.647973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.648683 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.648689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:10.648852 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:10.649388 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:10.649405 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:10.652308 (+ 2903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":246,"threads_started":1}
W20260809 04:57:10.652771 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:10.644528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:51607 (local address 127.29.243.254:35297)
0809 04:57:10.644746 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:10.644753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:10.644987 (+ 234us) server_negotiation.cc:407] Connection header received
0809 04:57:10.645167 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:10.645172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:10.645238 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:10.645317 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:10.646070 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.646799 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.648106 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:10.648542 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:10.648652 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:10.652534 (+ 3882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:51607: BlockingRecv error: recv got EOF from 127.29.243.194:51607 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 04:57:11 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:11 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:11.116737 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:11.108677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:51379)
0809 04:57:11.109190 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:11.109207 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:11.109341 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:11.109672 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:11.109677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:11.110086 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:11.110239 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:11.110248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.111312 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.111317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:11.112299 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:11.112311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.112990 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.112995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:11.113158 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:11.113751 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:11.113768 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:11.116507 (+ 2739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":282,"threads_started":1}
W20260809 04:57:11.116918 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:11.108758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:51379 (local address 127.29.243.254:35297)
0809 04:57:11.108954 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:11.108960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:11.109293 (+ 333us) server_negotiation.cc:407] Connection header received
0809 04:57:11.109459 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:11.109462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:11.109539 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:11.109620 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:11.110392 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.111153 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.112444 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.112846 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.112954 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:11.116683 (+ 3729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:51379: BlockingRecv error: recv got EOF from 127.29.243.195:51379 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 04:57:11 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:11 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:11.213896 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:11.205691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:45835)
0809 04:57:11.206233 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:11.206250 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:11.206382 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:11.206712 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:11.206717 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:11.207187 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:11.207406 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:11.207416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.208391 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.208396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:11.209334 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:11.209346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.210035 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.210040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:11.210200 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:11.210774 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:11.210792 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:11.213646 (+ 2854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":267,"threads_started":1}
W20260809 04:57:11.214026 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:11.205798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:45835 (local address 127.29.243.254:35297)
0809 04:57:11.205991 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:11.205998 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:11.206333 (+ 335us) server_negotiation.cc:407] Connection header received
0809 04:57:11.206506 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:11.206512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:11.206581 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:11.206659 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:11.207559 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.208250 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.209476 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.209900 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.210005 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:11.213836 (+ 3831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:45835: BlockingRecv error: recv got EOF from 127.29.243.193:45835 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 04:57:11 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:11 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:11.661700 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:11.653616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:50275)
0809 04:57:11.654122 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:11.654139 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:11.654252 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:11.654585 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:11.654588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:11.654988 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:11.655153 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:11.655163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.656235 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.656239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:11.657202 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:11.657216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.657860 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.657865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:11.658000 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:11.658582 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:11.658600 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:11.661445 (+ 2845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":235,"threads_started":1}
W20260809 04:57:11.661870 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:11.653768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:50275 (local address 127.29.243.254:35297)
0809 04:57:11.653934 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:11.653938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:11.654209 (+ 271us) server_negotiation.cc:407] Connection header received
0809 04:57:11.654337 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:11.654343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:11.654410 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:11.654491 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:11.655346 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.656117 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.657319 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:11.657718 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:11.657810 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:11.661649 (+ 3839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:50275: BlockingRecv error: recv got EOF from 127.29.243.194:50275 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 04:57:12 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:12 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:12.126331 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:12.117866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:59447)
0809 04:57:12.118432 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:12.118452 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:12.118578 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:12.118961 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:12.118966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:12.119470 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:12.119644 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:12.119653 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.120777 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.120781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:12.121700 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:12.121712 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.122418 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.122424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:12.122576 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:12.123102 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:12.123120 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:12.126071 (+ 2951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":286,"threads_started":1}
W20260809 04:57:12.126608 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:12.117968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:59447 (local address 127.29.243.254:35297)
0809 04:57:12.118197 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:12.118204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:12.118538 (+ 334us) server_negotiation.cc:407] Connection header received
0809 04:57:12.118679 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:12.118682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:12.118796 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:12.118912 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:12.119805 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.120619 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.121834 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.122267 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.122374 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:12.126327 (+ 3953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:59447: BlockingRecv error: recv got EOF from 127.29.243.195:59447 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 04:57:12 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:12 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:12.222905 13052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:12.214900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:59187)
0809 04:57:12.215502 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:12.215523 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:12.215674 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:12.216038 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:12.216044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:12.216446 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:12.216603 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:12.216613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.217624 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.217629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:12.218680 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:12.218694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.219358 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.219363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:12.219486 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:12.220021 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:12.220038 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:12.222688 (+ 2650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":306,"threads_started":1}
W20260809 04:57:12.223109 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:12.214985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:59187 (local address 127.29.243.254:35297)
0809 04:57:12.215216 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:12.215223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:12.215612 (+ 389us) server_negotiation.cc:407] Connection header received
0809 04:57:12.215792 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:12.215798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:12.215879 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:12.215983 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:12.216726 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.217502 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.218804 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.219212 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.219328 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:12.222876 (+ 3548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:59187: BlockingRecv error: recv got EOF from 127.29.243.193:59187 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 04:57:12 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:12 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:12.671705 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:12.662646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:38325)
0809 04:57:12.663213 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:12.663231 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:12.663420 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:12.663791 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:12.663796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:12.664258 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:12.664467 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:12.664479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.665550 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.665554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:12.666509 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:12.666525 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.667322 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.667330 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:12.667516 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:12.668171 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:12.668191 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:12.671296 (+ 3105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":309,"threads_started":1}
W20260809 04:57:12.671959 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:12.662778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:38325 (local address 127.29.243.254:35297)
0809 04:57:12.662990 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:12.662998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:12.663368 (+ 370us) server_negotiation.cc:407] Connection header received
0809 04:57:12.663564 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:12.663569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:12.663643 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:12.663738 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:12.664627 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.665390 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.666673 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:12.667125 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:12.667252 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:12.671670 (+ 4418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:38325: BlockingRecv error: recv got EOF from 127.29.243.194:38325 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 04:57:13 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:13 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:13.139150 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:13.127390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:41199)
0809 04:57:13.127940 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:13.127959 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:13.128101 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:13.128424 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:13.128428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:13.128845 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:13.129008 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:13.129018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.130022 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.130027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:13.130998 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:13.131012 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.131854 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.131862 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:13.132035 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:13.136031 (+ 3996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:13.136050 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:13.138935 (+ 2885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":292,"threads_started":1}
W20260809 04:57:13.139385 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:13.127485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:41199 (local address 127.29.243.254:35297)
0809 04:57:13.127659 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:13.127664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:13.128029 (+ 365us) server_negotiation.cc:407] Connection header received
0809 04:57:13.128199 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:13.128202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:13.128277 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:13.128358 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:13.129147 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.129877 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.131143 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.131623 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.131751 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:13.139122 (+ 7371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:41199: BlockingRecv error: recv got EOF from 127.29.243.195:41199 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 04:57:13 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:13 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:13.232797 13056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:13.223924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:45825)
0809 04:57:13.224535 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:13.224552 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:13.224676 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:13.224982 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:13.224985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:13.225398 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:13.225589 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:13.225599 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.226656 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.226660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:13.227976 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:13.227993 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.228883 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.228888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:13.229005 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:13.229602 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:13.229619 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:13.232442 (+ 2823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":269,"threads_started":1}
W20260809 04:57:13.232867 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:13.223993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:45825 (local address 127.29.243.254:35297)
0809 04:57:13.224218 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:13.224227 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:13.224658 (+ 431us) server_negotiation.cc:407] Connection header received
0809 04:57:13.224775 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:13.224779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:13.224858 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:13.224943 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:13.225757 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.226510 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.228141 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.228743 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.228838 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:13.232647 (+ 3809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:45825: BlockingRecv error: recv got EOF from 127.29.243.193:45825 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 04:57:13 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:13 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:13.681207 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:13.672746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:54697)
0809 04:57:13.673291 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:13.673307 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:13.673456 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:13.673814 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:13.673819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:13.674268 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:13.674447 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:13.674457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.675543 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.675547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:13.676482 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:13.676494 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.677191 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.677197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:13.677354 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:13.677921 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:13.677940 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:13.680856 (+ 2916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":238,"threads_started":1}
W20260809 04:57:13.681321 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:13.672841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:54697 (local address 127.29.243.254:35297)
0809 04:57:13.673040 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:13.673046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:13.673410 (+ 364us) server_negotiation.cc:407] Connection header received
0809 04:57:13.673584 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:13.673588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:13.673674 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:13.673756 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:13.674593 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.675406 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.676630 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:13.677040 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:13.677149 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:13.681079 (+ 3930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:54697: BlockingRecv error: recv got EOF from 127.29.243.194:54697 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 04:57:14 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:14 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:14.148847 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:14.140108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:51317)
0809 04:57:14.140710 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:14.140727 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:14.140878 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:14.141239 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:14.141245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:14.141709 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:14.141885 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:14.141896 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.142928 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.142932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:14.143870 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:14.143883 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.144618 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.144624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:14.144777 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:14.145362 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:14.145380 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:14.148586 (+ 3206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":291,"threads_started":1}
W20260809 04:57:14.148962 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:14.140185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:51317 (local address 127.29.243.254:35297)
0809 04:57:14.140367 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:14.140374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:14.140828 (+ 454us) server_negotiation.cc:407] Connection header received
0809 04:57:14.141009 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:14.141014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:14.141101 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:14.141185 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:14.142045 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.142782 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.144018 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.144466 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.144577 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:14.148778 (+ 4201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:51317: BlockingRecv error: recv got EOF from 127.29.243.195:51317 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 04:57:14 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:14 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:14.242723 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:14.233719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:41397)
0809 04:57:14.234508 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:14.234524 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:14.234647 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:14.234977 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:14.234981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:14.235464 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:14.235627 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:14.235637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.236678 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.236683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:14.237614 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:14.237626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.238486 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.238491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:14.238617 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:14.239210 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:14.239229 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:14.242398 (+ 3169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":514,"threads_started":1}
W20260809 04:57:14.242897 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:14.233948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:41397 (local address 127.29.243.254:35297)
0809 04:57:14.234155 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:14.234162 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:14.234620 (+ 458us) server_negotiation.cc:407] Connection header received
0809 04:57:14.234741 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:14.234745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:14.234813 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:14.234915 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:14.235767 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.236518 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.237758 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.238304 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.238420 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:14.242654 (+ 4234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:41397: BlockingRecv error: recv got EOF from 127.29.243.193:41397 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 04:57:14 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:14 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:14.690814 13061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:14.682208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:59583)
0809 04:57:14.682779 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:14.682794 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:14.682932 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:14.683263 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:14.683290 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:14.683724 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:14.683888 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:14.683897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.685070 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.685074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:14.685985 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:14.685997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.686737 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.686744 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:14.686878 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:14.687542 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:14.687565 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:14.690497 (+ 2932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":301,"threads_started":1}
W20260809 04:57:14.690963 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:14.682334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:59583 (local address 127.29.243.254:35297)
0809 04:57:14.682565 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:14.682572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:14.682882 (+ 310us) server_negotiation.cc:407] Connection header received
0809 04:57:14.683053 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:14.683059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:14.683126 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:14.683208 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:14.684035 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.684910 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.686125 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:14.686554 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:14.686663 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:14.690702 (+ 4039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:59583: BlockingRecv error: recv got EOF from 127.29.243.194:59583 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 04:57:15 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:15 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:15.158284 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:15.149923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:56975)
0809 04:57:15.150432 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:15.150449 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:15.150568 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:15.150833 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:15.150836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:15.151227 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:15.151420 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:15.151429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.152394 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.152398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:15.153315 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:15.153325 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.154086 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.154091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:15.154219 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:15.154810 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:15.154829 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:15.157963 (+ 3134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":248,"threads_started":1}
W20260809 04:57:15.158443 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:15.150007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:56975 (local address 127.29.243.254:35297)
0809 04:57:15.150210 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:15.150215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:15.150520 (+ 305us) server_negotiation.cc:407] Connection header received
0809 04:57:15.150655 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:15.150658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:15.150722 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:15.150804 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:15.151531 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.152254 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.153473 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.153906 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.154008 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:15.158186 (+ 4178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:56975: BlockingRecv error: recv got EOF from 127.29.243.195:56975 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 04:57:15 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:15 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:15.251886 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:15.243809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:59121)
0809 04:57:15.244318 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:15.244336 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:15.244455 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:15.244764 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:15.244768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:15.245174 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:15.245344 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:15.245354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.246377 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.246381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:15.247348 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:15.247360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.248052 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.248056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:15.248174 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:15.248757 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:15.248776 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:15.251641 (+ 2865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":275,"threads_started":1}
W20260809 04:57:15.252117 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:15.243854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:59121 (local address 127.29.243.254:35297)
0809 04:57:15.244090 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:15.244097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:15.244411 (+ 314us) server_negotiation.cc:407] Connection header received
0809 04:57:15.244557 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:15.244560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:15.244626 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:15.244705 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:15.245499 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.246243 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.247485 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.247934 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.248027 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:15.251873 (+ 3846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:59121: BlockingRecv error: recv got EOF from 127.29.243.193:59121 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 04:57:15 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:15 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:15.700291 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:15.691811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:41701)
0809 04:57:15.692337 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:15.692353 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:15.692509 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:15.692836 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:15.692841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:15.693266 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:15.693440 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:15.693450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.694511 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.694515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:15.695582 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:15.695596 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.696425 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.696431 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:15.696627 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:15.697230 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:15.697248 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:15.700054 (+ 2806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":261,"threads_started":1}
W20260809 04:57:15.700441 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:15.691837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:41701 (local address 127.29.243.254:35297)
0809 04:57:15.692036 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:15.692044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:15.692440 (+ 396us) server_negotiation.cc:407] Connection header received
0809 04:57:15.692633 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:15.692638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:15.692706 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:15.692786 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:15.693598 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.694359 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.695714 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:15.696249 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:15.696382 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:15.700232 (+ 3850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:41701: BlockingRecv error: recv got EOF from 127.29.243.194:41701 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 04:57:16 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:16 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:16.168079 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:16.159461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:33549)
0809 04:57:16.160025 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:16.160045 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:16.160198 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:16.160619 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:16.160623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:16.161175 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:16.161373 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:16.161387 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.162394 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.162398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:16.163366 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:16.163378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.164143 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.164148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:16.164283 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:16.164894 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:16.164913 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:16.167792 (+ 2879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":300,"threads_started":1}
W20260809 04:57:16.168186 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:16.159518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:33549 (local address 127.29.243.254:35297)
0809 04:57:16.159709 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:16.159717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:16.160156 (+ 439us) server_negotiation.cc:407] Connection header received
0809 04:57:16.160316 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:16.160323 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:16.160449 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:16.160551 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:16.161547 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.162278 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.163528 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.163972 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.164084 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:16.167991 (+ 3907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:33549: BlockingRecv error: recv got EOF from 127.29.243.195:33549 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 04:57:16 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:16 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:16.261302 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:16.252840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:57309)
0809 04:57:16.253376 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:16.253392 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:16.253512 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:16.253797 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:16.253800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:16.254177 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:16.254357 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:16.254372 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.255664 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.255670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:16.256786 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:16.256800 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.257476 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.257480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:16.257601 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:16.258157 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:16.258174 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:16.260932 (+ 2758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":303,"threads_started":1}
W20260809 04:57:16.261418 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:16.252963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:57309 (local address 127.29.243.254:35297)
0809 04:57:16.253139 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:16.253144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:16.253457 (+ 313us) server_negotiation.cc:407] Connection header received
0809 04:57:16.253620 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:16.253624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:16.253684 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:16.253761 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:16.254530 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.255503 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.256915 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.257361 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.257455 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:16.261167 (+ 3712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:57309: BlockingRecv error: recv got EOF from 127.29.243.193:57309 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 04:57:16 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:16 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:16.709944 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:16.701212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:54555)
0809 04:57:16.701774 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:16.701790 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:16.701909 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:16.702200 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:16.702203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:16.702577 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:16.702747 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:16.702760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.703890 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.703896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:16.704898 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:16.704913 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.705724 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.705730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:16.705921 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:16.706613 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:16.706636 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:16.709612 (+ 2976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":342,"threads_started":1}
W20260809 04:57:16.710096 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:16.701327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:54555 (local address 127.29.243.254:35297)
0809 04:57:16.701496 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:16.701508 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:16.701878 (+ 370us) server_negotiation.cc:407] Connection header received
0809 04:57:16.702010 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:16.702014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:16.702081 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:16.702165 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:16.702908 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.703714 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.705067 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:16.705569 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:16.705709 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:16.709831 (+ 4122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:54555: BlockingRecv error: recv got EOF from 127.29.243.194:54555 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 04:57:17 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:17 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:17.177680 13071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:17.169126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:41719)
0809 04:57:17.169618 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:17.169639 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:17.169781 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:17.170096 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:17.170101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:17.170587 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:17.170746 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:17.170755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.172128 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.172132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:17.173232 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:17.173247 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.173900 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.173904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:17.174013 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:17.174554 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:17.174572 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:17.177368 (+ 2796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":262,"threads_started":1}
W20260809 04:57:17.177815 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:17.169336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:41719 (local address 127.29.243.254:35297)
0809 04:57:17.169622 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:17.169629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:17.169722 (+ 93us) server_negotiation.cc:407] Connection header received
0809 04:57:17.169898 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:17.169902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:17.169965 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:17.170046 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:17.170920 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.171987 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.173359 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.173777 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.173870 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:17.177591 (+ 3721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:41719: BlockingRecv error: recv got EOF from 127.29.243.195:41719 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Aug 09 04:57:17 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:17 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:17.271217 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:17.262220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:45655)
0809 04:57:17.262765 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:17.262782 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:17.262929 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:17.263320 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:17.263323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:17.263740 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:17.263984 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:17.263998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.265211 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.265216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:17.266481 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:17.266497 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.267362 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.267368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:17.267551 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:17.268114 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:17.268134 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:17.270965 (+ 2831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":299,"threads_started":1}
W20260809 04:57:17.271584 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:17.262317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:45655 (local address 127.29.243.254:35297)
0809 04:57:17.262490 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:17.262495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:17.262845 (+ 350us) server_negotiation.cc:407] Connection header received
0809 04:57:17.263114 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:17.263118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:17.263187 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:17.263299 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:17.264154 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.265083 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.266633 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.267190 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.267333 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:17.271170 (+ 3837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:45655: BlockingRecv error: recv got EOF from 127.29.243.193:45655 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 04:57:17 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:17 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:17.719134 13074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:17.710868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:43121)
0809 04:57:17.711419 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:17.711437 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:17.711554 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:17.711825 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:17.711828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:17.712245 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:17.712423 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:17.712432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.713439 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.713443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:17.714338 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:17.714352 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.715049 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.715053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:17.715178 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:17.715765 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:17.715783 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:17.718873 (+ 3090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":282,"threads_started":1}
W20260809 04:57:17.719347 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:17.710958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:43121 (local address 127.29.243.254:35297)
0809 04:57:17.711120 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:17.711124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:17.711522 (+ 398us) server_negotiation.cc:407] Connection header received
0809 04:57:17.711646 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:17.711650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:17.711715 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:17.711796 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:17.712557 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.713313 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.714485 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:17.714907 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:17.715005 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:17.719098 (+ 4093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:43121: BlockingRecv error: recv got EOF from 127.29.243.194:43121 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 04:57:18 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:18 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:18.192189 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:18.178605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:46479)
0809 04:57:18.179079 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:18.179096 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:18.179214 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:18.179553 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:18.179556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:18.179960 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:18.180155 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:18.180164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.181266 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.181270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:18.182214 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:18.182248 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.183009 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.183015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:18.183195 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:18.188933 (+ 5738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:18.188953 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:18.191897 (+ 2944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":249,"threads_started":1}
W20260809 04:57:18.192301 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:18.178722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:46479 (local address 127.29.243.254:35297)
0809 04:57:18.178950 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:18.178956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:18.179193 (+ 237us) server_negotiation.cc:407] Connection header received
0809 04:57:18.179332 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:18.179336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:18.179408 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:18.179521 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:18.180335 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.181117 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.182373 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.182844 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.182951 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:18.192104 (+ 9153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:46479: BlockingRecv error: recv got EOF from 127.29.243.195:46479 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 04:57:18 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:18 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:18.280807 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:18.272220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:55629)
0809 04:57:18.272781 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:18.272798 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:18.272923 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:18.273213 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:18.273216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:18.273649 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:18.273828 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:18.273841 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.274908 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.274912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:18.275906 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:18.275920 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.276662 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.276667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:18.276805 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:18.277469 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:18.277488 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:18.280462 (+ 2974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":324,"threads_started":1}
W20260809 04:57:18.280910 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:18.272270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:55629 (local address 127.29.243.254:35297)
0809 04:57:18.272456 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:18.272461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:18.272868 (+ 407us) server_negotiation.cc:407] Connection header received
0809 04:57:18.273028 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:18.273031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:18.273095 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:18.273173 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:18.273970 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.274762 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.276038 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.276512 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.276605 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:18.280685 (+ 4080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:55629: BlockingRecv error: recv got EOF from 127.29.243.193:55629 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 04:57:18 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:18 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:18.728159 13080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:18.720126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:46835)
0809 04:57:18.720622 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:18.720643 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:18.720790 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:18.721146 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:18.721151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:18.721572 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:18.721732 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:18.721742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.722740 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.722746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:18.723697 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:18.723710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.724394 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.724399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:18.724581 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:18.725148 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:18.725165 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:18.727810 (+ 2645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"spinlock_wait_cycles":33536,"thread_start_us":344,"threads_started":1}
W20260809 04:57:18.728243 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:18.720226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:46835 (local address 127.29.243.254:35297)
0809 04:57:18.720443 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:18.720449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:18.720730 (+ 281us) server_negotiation.cc:407] Connection header received
0809 04:57:18.720924 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:18.720929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:18.720993 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:18.721072 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:18.721886 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.722593 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.723840 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:18.724254 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:18.724365 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:18.728028 (+ 3663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:46835: BlockingRecv error: recv got EOF from 127.29.243.194:46835 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 04:57:19 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:19 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:19.201627 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:19.193106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:41525)
0809 04:57:19.193623 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:19.193638 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:19.193773 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:19.194093 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:19.194097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:19.194514 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:19.194671 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:19.194680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.195736 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.195740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:19.196680 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:19.196693 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.197408 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.197414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:19.197587 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:19.198188 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:19.198205 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:19.201308 (+ 3103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":300,"threads_started":1}
W20260809 04:57:19.201781 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:19.193169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:41525 (local address 127.29.243.254:35297)
0809 04:57:19.193396 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:19.193432 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:19.193722 (+ 290us) server_negotiation.cc:407] Connection header received
0809 04:57:19.193888 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:19.193892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:19.193957 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:19.194038 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:19.194824 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.195588 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.196823 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.197271 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.197382 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:19.201547 (+ 4165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:41525: BlockingRecv error: recv got EOF from 127.29.243.195:41525 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 04:57:19 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:19 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:19.290596 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:19.281793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:45327)
0809 04:57:19.282368 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:19.282386 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:19.282528 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:19.282935 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:19.282941 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:19.283405 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:19.283586 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:19.283597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.284774 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.284779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:19.285747 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:19.285761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.286527 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.286533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:19.286686 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:19.287299 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:19.287319 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:19.290303 (+ 2984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":271,"threads_started":1}
W20260809 04:57:19.290786 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:19.281930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:45327 (local address 127.29.243.254:35297)
0809 04:57:19.282124 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:19.282129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:19.282480 (+ 351us) server_negotiation.cc:407] Connection header received
0809 04:57:19.282652 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:19.282658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:19.282755 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:19.282883 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:19.283725 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.284615 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.285910 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.286334 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.286437 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:19.290527 (+ 4090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:45327: BlockingRecv error: recv got EOF from 127.29.243.193:45327 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 04:57:19 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:19 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:19.737728 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:19.729113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:32777)
0809 04:57:19.729702 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:19.729718 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:19.729855 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:19.730181 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:19.730186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:19.730604 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:19.730775 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:19.730785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.731897 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.731901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:19.732821 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:19.732834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.733539 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.733545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:19.733693 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:19.734247 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:19.734269 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:19.737370 (+ 3101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":322,"threads_started":1}
W20260809 04:57:19.737802 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:19.729296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:32777 (local address 127.29.243.254:35297)
0809 04:57:19.729526 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:19.729533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:19.729802 (+ 269us) server_negotiation.cc:407] Connection header received
0809 04:57:19.729976 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:19.729980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:19.730045 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:19.730130 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:19.730940 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.731756 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.732965 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:19.733359 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:19.733468 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:19.737583 (+ 4115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:32777: BlockingRecv error: recv got EOF from 127.29.243.194:32777 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 04:57:20 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:20 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:20.210955 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:20.202598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:43257)
0809 04:57:20.203234 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:20.203256 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:20.203427 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:20.203794 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:20.203797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:20.204302 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:20.204538 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:20.204552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.205573 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.205577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:20.206483 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:20.206495 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.207204 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.207209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:20.207346 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:20.207911 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:20.207928 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:20.210696 (+ 2768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":353,"threads_started":1}
W20260809 04:57:20.211118 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:20.202719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:43257 (local address 127.29.243.254:35297)
0809 04:57:20.202916 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:20.202923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:20.203358 (+ 435us) server_negotiation.cc:407] Connection header received
0809 04:57:20.203570 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:20.203576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:20.203643 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:20.203743 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:20.204692 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.205450 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.206621 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.207035 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.207138 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:20.210922 (+ 3784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:43257: BlockingRecv error: recv got EOF from 127.29.243.195:43257 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 04:57:20 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:20 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:20.300720 13086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:20.291563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:41753)
0809 04:57:20.292130 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:20.292147 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:20.292296 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:20.292658 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:20.292663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:20.293087 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:20.293289 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:20.293300 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.294451 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.294455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:20.295469 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:20.295483 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.296253 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.296259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:20.296445 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:20.297097 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:20.297117 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:20.300429 (+ 3312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":290,"threads_started":1}
W20260809 04:57:20.300807 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:20.291608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:41753 (local address 127.29.243.254:35297)
0809 04:57:20.292005 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:20.292012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:20.292230 (+ 218us) server_negotiation.cc:407] Connection header received
0809 04:57:20.292445 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:20.292450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:20.292518 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:20.292607 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:20.293477 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.294283 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.295624 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.296090 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.296211 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:20.300612 (+ 4401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:41753: BlockingRecv error: recv got EOF from 127.29.243.193:41753 (error 108)
Metrics: {"server-negotiator.queue_time_us":205}
Aug 09 04:57:20 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:20 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:20.747002 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:20.738653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:41641)
0809 04:57:20.739149 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:20.739166 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:20.739300 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:20.739641 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:20.739645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:20.740103 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:20.740297 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:20.740308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.741663 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.741668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:20.742594 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:20.742606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.743326 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.743333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:20.743461 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:20.744061 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:20.744080 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:20.746752 (+ 2672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":252,"threads_started":1}
W20260809 04:57:20.747167 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:20.738782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:41641 (local address 127.29.243.254:35297)
0809 04:57:20.738950 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:20.738954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:20.739228 (+ 274us) server_negotiation.cc:407] Connection header received
0809 04:57:20.739448 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:20.739452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:20.739518 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:20.739623 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:20.740454 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.741530 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.742721 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:20.743158 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:20.743253 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:20.746948 (+ 3695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:41641: BlockingRecv error: recv got EOF from 127.29.243.194:41641 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 04:57:21 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:21 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.195@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:21.220039 13088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:21.211936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.195:45639)
0809 04:57:21.212393 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:21.212410 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:21.212530 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:21.212829 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:21.212833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:21.213222 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:21.213411 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:21.213419 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.214385 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.214392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:21.215618 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:21.215632 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.216348 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.216352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:21.216483 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:21.217044 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:21.217061 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:21.219782 (+ 2721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":238,"threads_started":1}
W20260809 04:57:21.220175 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:21.212077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:45639 (local address 127.29.243.254:35297)
0809 04:57:21.212247 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:21.212252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:21.212487 (+ 235us) server_negotiation.cc:407] Connection header received
0809 04:57:21.212632 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:21.212635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:21.212718 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:21.212793 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:21.213528 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.214221 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.215761 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.216219 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.216316 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:21.219988 (+ 3672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.195:45639: BlockingRecv error: recv got EOF from 127.29.243.195:45639 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 04:57:21 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:21 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.193@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:21.310328 13089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:21.302063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.193:48987)
0809 04:57:21.302548 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:21.302564 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:21.302678 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:21.302961 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:21.302965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:21.303426 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:21.303620 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:21.303630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.304739 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.304742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:21.305672 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:21.305685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.306303 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.306307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:21.306415 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:21.306982 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:21.306999 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:21.309998 (+ 2999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":251,"threads_started":1}
W20260809 04:57:21.310406 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:21.302156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:48987 (local address 127.29.243.254:35297)
0809 04:57:21.302458 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:21.302464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:21.302630 (+ 166us) server_negotiation.cc:407] Connection header received
0809 04:57:21.302765 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:21.302768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:21.302839 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:21.302933 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:21.303773 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.304609 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.305803 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.306196 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.306289 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:21.310187 (+ 3898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.193:48987: BlockingRecv error: recv got EOF from 127.29.243.193:48987 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 04:57:21 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:57:21 dist-test-slave-21b9 krb5kdc[11486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.243.194@KRBTEST.COM for oryx/127.29.243.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:57:21.756472 13090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:21.748103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:35297 (local address 127.29.243.194:34057)
0809 04:57:21.748653 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:21.748669 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:21.748802 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:21.749156 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:21.749160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:21.749603 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:21.749777 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:21.749787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.750831 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.750838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:21.751819 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:21.751846 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.752771 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.752777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:21.752960 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:21.753479 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:21.753495 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:21.756219 (+ 2724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.243.254:35297: Server oryx/127.29.243.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":296,"threads_started":1}
W20260809 04:57:21.756702 12623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:21.748261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:34057 (local address 127.29.243.254:35297)
0809 04:57:21.748501 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:21.748513 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:21.748749 (+ 236us) server_negotiation.cc:407] Connection header received
0809 04:57:21.748932 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:21.748938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:21.749010 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:21.749105 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:21.749939 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.750676 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.751965 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:21.752637 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:21.752753 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:21.756460 (+ 3707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.243.194:34057: BlockingRecv error: recv got EOF from 127.29.243.194:34057 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 04:57:22.056793 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 12391
I20260809 04:57:22.069144 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:22.057900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.29.243.193:43019)
0809 04:57:22.058529 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:22.058537 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:22.058566 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:57:22.058639 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:22.058647 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:22.058874 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:22.059056 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:22.060539 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:22.061597 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:22.062827 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:22.063480 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:22.063594 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:22.066067 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:22.066076 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:22.066092 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:22.066125 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:22.068613 (+ 2488us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.243.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 04:57:22.068882 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35628: Request ticket server kudu/127.29.243.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":322,"threads_started":1}
W20260809 04:57:22.069244 13094 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35628: Request ticket server kudu/127.29.243.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 04:57:22.069371 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 12391: Not authorized: Client connection negotiation failed: client connection to 127.29.243.193:43019: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.243.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 04:57:22.069594 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 12391
I20260809 04:57:22.098430 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 12625
I20260809 04:57:22.110699 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:22.099567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58432 (local address 127.29.243.194:40773)
0809 04:57:22.100193 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:22.100201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:22.100234 (+ 33us) server_negotiation.cc:407] Connection header received
0809 04:57:22.100303 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:22.100310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:22.100538 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:22.100713 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:22.101906 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:22.103032 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:22.104394 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:22.105003 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:22.105127 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:22.107486 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:22.107496 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:22.107511 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:22.107546 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:22.110150 (+ 2604us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.243.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 04:57:22.110437 (+ 287us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58432: Request ticket server kudu/127.29.243.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":298,"threads_started":1}
W20260809 04:57:22.110759 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 12625: Not authorized: Client connection negotiation failed: client connection to 127.29.243.194:40773: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.243.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 04:57:22.110811 13095 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58432: Request ticket server kudu/127.29.243.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 04:57:22.110917 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 12625
I20260809 04:57:22.136127 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 12830
I20260809 04:57:22.147910 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:22.137139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56302 (local address 127.29.243.195:35343)
0809 04:57:22.137763 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:22.137771 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:22.137799 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:57:22.137870 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:22.137876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:22.138135 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:22.138294 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:22.139733 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:22.140811 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:22.141962 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:22.142489 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:22.142584 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:22.144934 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:22.144942 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:22.144957 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:22.144997 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:22.147426 (+ 2429us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.243.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 04:57:22.147669 (+ 243us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56302: Request ticket server kudu/127.29.243.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":353,"threads_started":1}
W20260809 04:57:22.148028 13096 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56302: Request ticket server kudu/127.29.243.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 04:57:22.148031 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 12830: Not authorized: Client connection negotiation failed: client connection to 127.29.243.195:35343: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.243.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 04:57:22.148185 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 12830
I20260809 04:57:22.175361 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 12287
I20260809 04:57:22.176795 12381 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:22.295042 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 12287
2026-08-09T04:57:22Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21410 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T04:57:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:57:22Z Disabled control of system clock
I20260809 04:57:22.388527 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:36545
--webserver_interface=127.29.243.254
--webserver_port=0
--builtin_ntp_servers=127.29.243.212:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:36545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 04:57:22.725651 13102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:22.725943 13102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:22.726053 13102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:22.733265 13102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:57:22.733366 13102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:22.733428 13102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:22.733517 13102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:57:22.733577 13102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:57:22.742596 13102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:36545
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:36545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:22.743949 13102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:22.745747 13102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:22.757349 13110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:22.757440 13109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:22.757422 13113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:22.758100 13102 server_base.cc:1034] running on GCE node
I20260809 04:57:22.758917 13102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:22.760425 13102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:22.761637 13102 hybrid_clock.cc:648] HybridClock initialized: now 1786251442761585 us; error 43 us; skew 500 ppm
I20260809 04:57:22.761988 13102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:57:22.764230 13102 webserver.cc:460] Webserver started at http://127.29.243.254:41199/ using document root <none> and password file <none>
I20260809 04:57:22.764951 13102 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:22.765051 13102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:22.765370 13102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:22.768309 13102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "4c6dee50359f4530bd8124d98e02c642"
format_stamp: "Formatted at 2026-08-09 04:57:22 on dist-test-slave-21b9"
server_key: "af0c16d4cd3f29dbc429702971c18784"
server_key_iv: "9a7a8043f3ae306d402db4df6d53e338"
server_key_version: "encryptionkey@0"
I20260809 04:57:22.769115 13102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "4c6dee50359f4530bd8124d98e02c642"
format_stamp: "Formatted at 2026-08-09 04:57:22 on dist-test-slave-21b9"
server_key: "af0c16d4cd3f29dbc429702971c18784"
server_key_iv: "9a7a8043f3ae306d402db4df6d53e338"
server_key_version: "encryptionkey@0"
I20260809 04:57:22.774298 13102 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 04:57:22.777668 13122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:22.779031 13102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:57:22.779212 13102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "4c6dee50359f4530bd8124d98e02c642"
format_stamp: "Formatted at 2026-08-09 04:57:22 on dist-test-slave-21b9"
server_key: "af0c16d4cd3f29dbc429702971c18784"
server_key_iv: "9a7a8043f3ae306d402db4df6d53e338"
server_key_version: "encryptionkey@0"
I20260809 04:57:22.779500 13102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:22.803191 13102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:22.804086 13102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:22.804359 13102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:22.821909 13102 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:36545
I20260809 04:57:22.821945 13200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:36545 every 8 connection(s)
I20260809 04:57:22.823256 13102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:57:22.827144 13202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:57:22.831948 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 13102
I20260809 04:57:22.832185 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:57:22.833069 30671 external_mini_cluster.cc:1428] Setting key 85263cfee71503f1ee035a035bebadae
I20260809 04:57:22.838807 13202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642: Bootstrap starting.
I20260809 04:57:22.841856 13202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642: Neither blocks nor log segments found. Creating new log.
I20260809 04:57:22.842888 13202 log.cc:824] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642: Log is configured to *not* fsync() on all Append() calls
I20260809 04:57:22.846207 13202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642: No bootstrap required, opened a new log
I20260809 04:57:22.846832 13205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:22.835237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57468 (local address 127.29.243.254:36545)
0809 04:57:22.836276 (+ 1039us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:22.836290 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:22.836333 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:57:22.837975 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:22.837999 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:22.838420 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:22.838775 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:22.839422 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:22.840709 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:22.842022 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:22.842690 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:22.842871 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:22.844064 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:22.844072 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:22.844086 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:22.844780 (+ 694us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:22.844879 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:22.844904 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:22.845020 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:22.845240 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 04:57:22.845575 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":358,"threads_started":1}
I20260809 04:57:22.853693 13202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6dee50359f4530bd8124d98e02c642" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36545 } }
I20260809 04:57:22.854027 13202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:57:22.854117 13202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c6dee50359f4530bd8124d98e02c642, State: Initialized, Role: FOLLOWER
I20260809 04:57:22.854602 13202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6dee50359f4530bd8124d98e02c642" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36545 } }
I20260809 04:57:22.854795 13202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:57:22.854889 13202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:57:22.855028 13202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:57:22.857378 13202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6dee50359f4530bd8124d98e02c642" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36545 } }
I20260809 04:57:22.857828 13202 leader_election.cc:302] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c6dee50359f4530bd8124d98e02c642; no voters:
I20260809 04:57:22.858393 13202 leader_election.cc:288] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:57:22.858569 13208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:57:22.859244 13208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [term 1 LEADER]: Becoming Leader. State: Replica: 4c6dee50359f4530bd8124d98e02c642, State: Running, Role: LEADER
I20260809 04:57:22.859804 13208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6dee50359f4530bd8124d98e02c642" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36545 } }
I20260809 04:57:22.860517 13202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:57:22.862136 13211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c6dee50359f4530bd8124d98e02c642. Latest consensus state: current_term: 1 leader_uuid: "4c6dee50359f4530bd8124d98e02c642" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6dee50359f4530bd8124d98e02c642" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36545 } } }
I20260809 04:57:22.862583 13211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [sys.catalog]: This master's current role is: LEADER
I20260809 04:57:22.863133 13210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c6dee50359f4530bd8124d98e02c642" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6dee50359f4530bd8124d98e02c642" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36545 } } }
I20260809 04:57:22.864771 13210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642 [sys.catalog]: This master's current role is: LEADER
I20260809 04:57:22.868599 13216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:57:22.873337 13216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:57:22.881629 13216 catalog_manager.cc:1347] Generated new cluster ID: 606f5e0a55d440f0b8d9d51fff4b6198
I20260809 04:57:22.881734 13216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:57:22.892760 13216 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:57:22.894594 13216 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:57:22.905579 13216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c6dee50359f4530bd8124d98e02c642: Generated new TSK 0
I20260809 04:57:22.906594 13216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:57:22.920248 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36545
--builtin_ntp_servers=127.29.243.212:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 04:57:23.280299 13240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:23.280587 13240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:23.280699 13240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:23.287972 13240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:23.288069 13240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:23.288210 13240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:57:23.297449 13240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:23.298905 13240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:23.300760 13240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:23.313339 13246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:23.313391 13249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:23.313369 13247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:23.314354 13240 server_base.cc:1034] running on GCE node
I20260809 04:57:23.314960 13240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:23.315927 13240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:23.317135 13240 hybrid_clock.cc:648] HybridClock initialized: now 1786251443317082 us; error 40 us; skew 500 ppm
I20260809 04:57:23.317445 13240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:57:23.324155 13240 webserver.cc:460] Webserver started at http://127.29.243.193:37167/ using document root <none> and password file <none>
I20260809 04:57:23.324905 13240 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:23.325006 13240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:23.325331 13240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:23.328248 13240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "bdb73ba3bd4d41bebae8e9716213f659"
format_stamp: "Formatted at 2026-08-09 04:57:23 on dist-test-slave-21b9"
server_key: "e53d60f512c4f945f875b7fab3699c7a"
server_key_iv: "3ccffa4a4ba3c85b581d5916d2738707"
server_key_version: "encryptionkey@0"
I20260809 04:57:23.329020 13240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "bdb73ba3bd4d41bebae8e9716213f659"
format_stamp: "Formatted at 2026-08-09 04:57:23 on dist-test-slave-21b9"
server_key: "e53d60f512c4f945f875b7fab3699c7a"
server_key_iv: "3ccffa4a4ba3c85b581d5916d2738707"
server_key_version: "encryptionkey@0"
I20260809 04:57:23.334110 13240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:57:23.337543 13255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:23.338790 13240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 04:57:23.338938 13240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "bdb73ba3bd4d41bebae8e9716213f659"
format_stamp: "Formatted at 2026-08-09 04:57:23 on dist-test-slave-21b9"
server_key: "e53d60f512c4f945f875b7fab3699c7a"
server_key_iv: "3ccffa4a4ba3c85b581d5916d2738707"
server_key_version: "encryptionkey@0"
I20260809 04:57:23.339133 13240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:23.363623 13240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:23.364504 13240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:23.364807 13240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:23.365780 13240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:23.367301 13240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:23.367415 13240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:23.367525 13240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:23.367600 13240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:23.398602 13240 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:41195
I20260809 04:57:23.398675 13429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:41195 every 8 connection(s)
I20260809 04:57:23.399943 13240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:57:23.403442 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 13240
I20260809 04:57:23.403632 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:57:23.404278 30671 external_mini_cluster.cc:1428] Setting key cf174adf38eed36fd25f9dd09943b650
I20260809 04:57:23.407321 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36545
--builtin_ntp_servers=127.29.243.212:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 04:57:23.412906 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:23.402574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:35627 (local address 127.29.243.254:36545)
0809 04:57:23.403031 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:23.403038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:23.404482 (+ 1444us) server_negotiation.cc:407] Connection header received
0809 04:57:23.406479 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:23.406485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:23.406568 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:23.406692 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:23.407904 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:23.408613 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:23.410181 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:23.410585 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:23.410693 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:23.412026 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:23.412032 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:23.412035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:23.412054 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:23.412093 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:23.412105 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:23.412176 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:23.412434 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 04:57:23.412535 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":202,"threads_started":1}
I20260809 04:57:23.413838 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:23.403233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:36545 (local address 127.29.243.193:35627)
0809 04:57:23.404317 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:23.404362 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:23.406219 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:23.406931 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:23.406939 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:23.407035 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:23.407713 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:23.407734 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:23.408770 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:23.408773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:23.410027 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:23.410040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:23.410806 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:23.410814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:23.411043 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:23.411713 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:57:23.411716 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:23.411754 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:57:23.411836 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:23.412191 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:23.412203 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:23.412210 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 04:57:23.412357 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 04:57:23.412760 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":319,"threads_started":1}
I20260809 04:57:23.415403 13430 heartbeater.cc:347] Connected to a master server at 127.29.243.254:36545
I20260809 04:57:23.415740 13430 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:23.416648 13430 heartbeater.cc:511] Master 127.29.243.254:36545 requested a full tablet report, sending...
I20260809 04:57:23.418880 13142 ts_manager.cc:194] Registered new tserver with Master: bdb73ba3bd4d41bebae8e9716213f659 (127.29.243.193:41195)
I20260809 04:57:23.420398 13142 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.193:35627
W20260809 04:57:23.748104 13437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:23.748399 13437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:23.748512 13437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:23.755565 13437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:23.755656 13437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:23.755784 13437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:57:23.764765 13437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:23.766049 13437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:23.768024 13437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:23.780817 13446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:23.780819 13445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:23.780810 13449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:23.781533 13437 server_base.cc:1034] running on GCE node
I20260809 04:57:23.782167 13437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:23.783212 13437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:23.784446 13437 hybrid_clock.cc:648] HybridClock initialized: now 1786251443784384 us; error 53 us; skew 500 ppm
I20260809 04:57:23.784814 13437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:57:23.787189 13437 webserver.cc:460] Webserver started at http://127.29.243.194:39945/ using document root <none> and password file <none>
I20260809 04:57:23.787921 13437 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:23.788017 13437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:23.788334 13437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:23.791139 13437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "93cc8f3de5d34b32be425c33779c8c2d"
format_stamp: "Formatted at 2026-08-09 04:57:23 on dist-test-slave-21b9"
server_key: "7785249b73f22141b6cd2029812ac117"
server_key_iv: "0f67d26aa985456709778636894c0601"
server_key_version: "encryptionkey@0"
I20260809 04:57:23.791944 13437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "93cc8f3de5d34b32be425c33779c8c2d"
format_stamp: "Formatted at 2026-08-09 04:57:23 on dist-test-slave-21b9"
server_key: "7785249b73f22141b6cd2029812ac117"
server_key_iv: "0f67d26aa985456709778636894c0601"
server_key_version: "encryptionkey@0"
I20260809 04:57:23.796962 13437 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:57:23.800405 13457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:23.801690 13437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:57:23.801831 13437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "93cc8f3de5d34b32be425c33779c8c2d"
format_stamp: "Formatted at 2026-08-09 04:57:23 on dist-test-slave-21b9"
server_key: "7785249b73f22141b6cd2029812ac117"
server_key_iv: "0f67d26aa985456709778636894c0601"
server_key_version: "encryptionkey@0"
I20260809 04:57:23.802021 13437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:23.839634 13437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:23.840682 13437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:23.841024 13437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:23.842182 13437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:23.844025 13437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:23.844151 13437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:23.844244 13437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:23.844301 13437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:23.874361 13437 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:45741
I20260809 04:57:23.874421 13631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:45741 every 8 connection(s)
I20260809 04:57:23.875792 13437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:57:23.877415 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 13437
I20260809 04:57:23.877606 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:57:23.878162 30671 external_mini_cluster.cc:1428] Setting key 5daf0eb159d80b6b9ce70a03ab00eb3d
I20260809 04:57:23.881290 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36545
--builtin_ntp_servers=127.29.243.212:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 04:57:23.888772 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:23.878484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:36491 (local address 127.29.243.254:36545)
0809 04:57:23.878675 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:23.878680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:23.880533 (+ 1853us) server_negotiation.cc:407] Connection header received
0809 04:57:23.882370 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:23.882376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:23.882467 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:23.882571 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:23.883654 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:23.884622 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:23.886172 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:23.886680 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:23.886799 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:23.888033 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:23.888039 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:23.888043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:23.888065 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:23.888103 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:23.888118 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:23.888197 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:23.888381 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 04:57:23.888486 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:57:23.889724 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:23.879233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:36545 (local address 127.29.243.194:36491)
0809 04:57:23.880374 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:23.880422 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:23.882092 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:23.882765 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:23.882772 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:23.882854 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:23.883484 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:23.883504 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:23.884751 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:23.884755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:23.886036 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:23.886049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:23.886863 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:23.886868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:23.887057 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:23.887721 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:57:23.887723 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:23.887763 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:57:23.887845 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:23.888204 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:23.888211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:23.888216 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:57:23.888338 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 04:57:23.888641 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":321,"threads_started":1}
I20260809 04:57:23.891014 13633 heartbeater.cc:347] Connected to a master server at 127.29.243.254:36545
I20260809 04:57:23.891340 13633 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:23.892220 13633 heartbeater.cc:511] Master 127.29.243.254:36545 requested a full tablet report, sending...
I20260809 04:57:23.893682 13142 ts_manager.cc:194] Registered new tserver with Master: 93cc8f3de5d34b32be425c33779c8c2d (127.29.243.194:45741)
I20260809 04:57:23.894469 13142 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.194:36491
W20260809 04:57:24.228600 13638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:24.228919 13638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:24.229511 13638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:24.236861 13638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:24.236980 13638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:24.237139 13638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:57:24.246389 13638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:24.247830 13638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:24.249750 13638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:24.262778 13647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:24.262813 13653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:24.262847 13648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:24.264133 13638 server_base.cc:1034] running on GCE node
I20260809 04:57:24.264771 13638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:24.265739 13638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:24.267045 13638 hybrid_clock.cc:648] HybridClock initialized: now 1786251444266974 us; error 58 us; skew 500 ppm
I20260809 04:57:24.267408 13638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:57:24.269718 13638 webserver.cc:460] Webserver started at http://127.29.243.195:40077/ using document root <none> and password file <none>
I20260809 04:57:24.270411 13638 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:24.270514 13638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:24.270860 13638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:24.273722 13638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "b019a31c83db46478c5badc4a308abb2"
format_stamp: "Formatted at 2026-08-09 04:57:24 on dist-test-slave-21b9"
server_key: "2d6f10d1611549f63a386f5fb6081f63"
server_key_iv: "9226309353da4f3f24cb69d007e3df2e"
server_key_version: "encryptionkey@0"
I20260809 04:57:24.274490 13638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "b019a31c83db46478c5badc4a308abb2"
format_stamp: "Formatted at 2026-08-09 04:57:24 on dist-test-slave-21b9"
server_key: "2d6f10d1611549f63a386f5fb6081f63"
server_key_iv: "9226309353da4f3f24cb69d007e3df2e"
server_key_version: "encryptionkey@0"
I20260809 04:57:24.279806 13638 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 04:57:24.283206 13662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:24.284577 13638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:57:24.284742 13638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "b019a31c83db46478c5badc4a308abb2"
format_stamp: "Formatted at 2026-08-09 04:57:24 on dist-test-slave-21b9"
server_key: "2d6f10d1611549f63a386f5fb6081f63"
server_key_iv: "9226309353da4f3f24cb69d007e3df2e"
server_key_version: "encryptionkey@0"
I20260809 04:57:24.284940 13638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:24.303138 13638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:24.304005 13638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:24.304291 13638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:24.305223 13638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:24.306768 13638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:24.306860 13638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:24.306939 13638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:24.306993 13638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:24.338024 13638 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:40129
I20260809 04:57:24.338063 13833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:40129 every 8 connection(s)
I20260809 04:57:24.339478 13638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:57:24.349723 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 13638
I20260809 04:57:24.349942 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:57:24.350468 30671 external_mini_cluster.cc:1428] Setting key 07453afb4b3f63dc101245759c223549
I20260809 04:57:24.351686 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:24.342073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:42345 (local address 127.29.243.254:36545)
0809 04:57:24.342280 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:24.342284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:24.343641 (+ 1357us) server_negotiation.cc:407] Connection header received
0809 04:57:24.345372 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:24.345376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:24.345448 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:24.345532 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:24.346509 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:24.347233 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:24.348769 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:24.349184 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:24.349274 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:24.350791 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:24.350796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:24.350800 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:24.350819 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:24.350852 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:24.350864 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:24.350931 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:24.351185 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 04:57:24.351305 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 04:57:24.352691 13837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:24.342539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:36545 (local address 127.29.243.195:42345)
0809 04:57:24.343485 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:24.343526 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:24.345144 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:24.345723 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:24.345730 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:24.345813 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:24.346376 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:24.346392 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:24.347366 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:24.347370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:24.348647 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:24.348664 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:24.349409 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:24.349418 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:24.349695 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:24.350434 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:57:24.350437 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:24.350491 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:57:24.350591 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:24.350963 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:24.350975 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:24.350983 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:57:24.351143 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 04:57:24.351555 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":264,"threads_started":1}
I20260809 04:57:24.354094 13835 heartbeater.cc:347] Connected to a master server at 127.29.243.254:36545
I20260809 04:57:24.354393 13835 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:24.355182 13835 heartbeater.cc:511] Master 127.29.243.254:36545 requested a full tablet report, sending...
I20260809 04:57:24.357172 13142 ts_manager.cc:194] Registered new tserver with Master: b019a31c83db46478c5badc4a308abb2 (127.29.243.195:40129)
I20260809 04:57:24.357944 13142 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.195:42345
I20260809 04:57:24.366071 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:57:24.372277 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:24.370582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57474 (local address 127.29.243.254:36545)
0809 04:57:24.370768 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:24.370773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:24.371303 (+ 530us) server_negotiation.cc:407] Connection header received
0809 04:57:24.371481 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:24.371485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:24.371570 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:24.371654 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:24.372100 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57474: BlockingRecv error: recv got EOF from 127.0.0.1:57474 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:57:24.373167 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 13240
I20260809 04:57:24.380746 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:24.374066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59600 (local address 127.29.243.193:41195)
0809 04:57:24.374597 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:24.374604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:24.374632 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:57:24.374689 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:24.374696 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:24.374910 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:24.375041 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:24.375373 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:24.376470 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:24.377716 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:24.378279 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:24.378421 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:24.379369 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:24.379378 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:24.379413 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:24.380018 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:24.380067 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:24.380088 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:24.380173 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:24.380315 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 04:57:24.380460 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":238,"threads_started":1}
I20260809 04:57:24.381997 13424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:24.461292 13430 heartbeater.cc:503] Master 127.29.243.254:36545 was elected leader, sending a full tablet report...
I20260809 04:57:24.463590 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 13240
I20260809 04:57:24.494046 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 13437
I20260809 04:57:24.502130 13854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:24.495000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60840 (local address 127.29.243.194:45741)
0809 04:57:24.495708 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:24.495715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:24.495746 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:57:24.495826 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:24.495837 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:24.496150 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:24.496286 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:24.496697 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:24.497731 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:24.498961 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:24.499620 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:24.499730 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:24.500661 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:24.500670 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:24.500684 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:24.501317 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:24.501364 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:24.501384 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:24.501475 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:24.501622 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 04:57:24.501748 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":428,"threads_started":1}
I20260809 04:57:24.503432 13626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:24.584218 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 13437
I20260809 04:57:24.610364 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 13638
I20260809 04:57:24.618256 13857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:24.611475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.29.243.195:40129)
0809 04:57:24.612126 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:24.612133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:24.612164 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:57:24.612228 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:24.612235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:24.612458 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:24.612634 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:24.613031 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:24.614067 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:24.615299 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:24.615868 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:24.615961 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:24.616894 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:24.616903 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:24.616917 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:24.617504 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:24.617553 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:24.617574 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:24.617661 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:24.617753 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 04:57:24.617890 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":324,"threads_started":1}
I20260809 04:57:24.619563 13828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:24.699863 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 13638
I20260809 04:57:24.725121 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 13102
I20260809 04:57:24.726363 13194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:24.830034 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 13102
2026-08-09T04:57:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2533 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T04:57:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:57:24Z Disabled control of system clock
I20260809 04:57:24.920759 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:36661
--webserver_interface=127.29.243.254
--webserver_port=0
--builtin_ntp_servers=127.29.243.212:38357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:36661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 04:57:25.276811 13865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:25.277105 13865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:25.277230 13865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:25.284483 13865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:57:25.284590 13865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:25.284634 13865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:25.284716 13865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:57:25.284773 13865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:57:25.293972 13865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:36661
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:36661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:25.295404 13865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:25.297338 13865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:25.309110 13871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:25.309150 13872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:25.309077 13875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:25.309906 13865 server_base.cc:1034] running on GCE node
I20260809 04:57:25.310673 13865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:25.312134 13865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:25.313344 13865 hybrid_clock.cc:648] HybridClock initialized: now 1786251445313298 us; error 36 us; skew 500 ppm
I20260809 04:57:25.313699 13865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:57:25.316161 13865 webserver.cc:460] Webserver started at http://127.29.243.254:39733/ using document root <none> and password file <none>
I20260809 04:57:25.316862 13865 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:25.316956 13865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:25.317284 13865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:25.320281 13865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "760a9391046a47fcbb1700402d62d52f"
format_stamp: "Formatted at 2026-08-09 04:57:25 on dist-test-slave-21b9"
server_key: "e0d12332fc4989b07df69b7283caa88f"
server_key_iv: "d2c7585aebe026fcec945daaadcbecf0"
server_key_version: "encryptionkey@0"
I20260809 04:57:25.321051 13865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "760a9391046a47fcbb1700402d62d52f"
format_stamp: "Formatted at 2026-08-09 04:57:25 on dist-test-slave-21b9"
server_key: "e0d12332fc4989b07df69b7283caa88f"
server_key_iv: "d2c7585aebe026fcec945daaadcbecf0"
server_key_version: "encryptionkey@0"
I20260809 04:57:25.326264 13865 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:57:25.329704 13882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:25.331009 13865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:57:25.331168 13865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "760a9391046a47fcbb1700402d62d52f"
format_stamp: "Formatted at 2026-08-09 04:57:25 on dist-test-slave-21b9"
server_key: "e0d12332fc4989b07df69b7283caa88f"
server_key_iv: "d2c7585aebe026fcec945daaadcbecf0"
server_key_version: "encryptionkey@0"
I20260809 04:57:25.331434 13865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:25.336856 13865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:25.337605 13865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:25.337874 13865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:25.354058 13865 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:36661
I20260809 04:57:25.354074 13974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:36661 every 8 connection(s)
I20260809 04:57:25.355453 13865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:57:25.356570 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 13865
I20260809 04:57:25.356794 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:57:25.357635 30671 external_mini_cluster.cc:1428] Setting key cafb0918d663a39a57dcb158a9e082a5
I20260809 04:57:25.360282 13975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:57:25.365972 13980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:25.359795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.29.243.254:36661)
0809 04:57:25.360852 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:25.360863 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:25.360905 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:57:25.362497 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:25.362520 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:25.362934 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:25.363302 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:25.364372 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:25.364378 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:25.364422 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:25.364436 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:25.364522 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:25.364540 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:25.364713 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:25.364812 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 04:57:25.365068 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":389,"threads_started":1}
I20260809 04:57:25.371933 13975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f: Bootstrap starting.
I20260809 04:57:25.374953 13975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f: Neither blocks nor log segments found. Creating new log.
I20260809 04:57:25.375972 13975 log.cc:824] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:57:25.378921 13975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f: No bootstrap required, opened a new log
I20260809 04:57:25.386188 13975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760a9391046a47fcbb1700402d62d52f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36661 } }
I20260809 04:57:25.386574 13975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:57:25.386667 13975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 760a9391046a47fcbb1700402d62d52f, State: Initialized, Role: FOLLOWER
I20260809 04:57:25.387262 13975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760a9391046a47fcbb1700402d62d52f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36661 } }
I20260809 04:57:25.387543 13975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:57:25.387672 13975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:57:25.387835 13975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:57:25.389967 13975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760a9391046a47fcbb1700402d62d52f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36661 } }
I20260809 04:57:25.390463 13975 leader_election.cc:302] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 760a9391046a47fcbb1700402d62d52f; no voters:
I20260809 04:57:25.391043 13975 leader_election.cc:288] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:57:25.391176 13987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:57:25.391897 13987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [term 1 LEADER]: Becoming Leader. State: Replica: 760a9391046a47fcbb1700402d62d52f, State: Running, Role: LEADER
I20260809 04:57:25.392416 13987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760a9391046a47fcbb1700402d62d52f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36661 } }
I20260809 04:57:25.393158 13975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:57:25.395556 13989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 760a9391046a47fcbb1700402d62d52f. Latest consensus state: current_term: 1 leader_uuid: "760a9391046a47fcbb1700402d62d52f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760a9391046a47fcbb1700402d62d52f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36661 } } }
I20260809 04:57:25.396314 13989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [sys.catalog]: This master's current role is: LEADER
I20260809 04:57:25.395555 13988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "760a9391046a47fcbb1700402d62d52f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760a9391046a47fcbb1700402d62d52f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 36661 } } }
I20260809 04:57:25.402217 13988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f [sys.catalog]: This master's current role is: LEADER
I20260809 04:57:25.404150 14001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:57:25.408576 14001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:57:25.416512 14001 catalog_manager.cc:1347] Generated new cluster ID: 7f80d5e8c55e49e79ec07ab7fa203d94
I20260809 04:57:25.416623 14001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:57:25.460853 14001 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:57:25.461283 14001 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:57:25.466650 14001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 760a9391046a47fcbb1700402d62d52f: Generated new TSK 0
I20260809 04:57:25.467514 14001 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:57:25.474990 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36661
--builtin_ntp_servers=127.29.243.212:38357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 04:57:25.841780 14018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:25.842034 14018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:25.842123 14018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:25.849184 14018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:25.849277 14018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:25.849381 14018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:57:25.858145 14018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:25.859549 14018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:25.861342 14018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:25.874012 14025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:25.873975 14026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:25.874044 14028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:25.874730 14018 server_base.cc:1034] running on GCE node
I20260809 04:57:25.875358 14018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:25.876255 14018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:25.877470 14018 hybrid_clock.cc:648] HybridClock initialized: now 1786251445877405 us; error 47 us; skew 500 ppm
I20260809 04:57:25.877800 14018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:57:25.880075 14018 webserver.cc:460] Webserver started at http://127.29.243.193:33563/ using document root <none> and password file <none>
I20260809 04:57:25.880790 14018 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:25.880890 14018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:25.881209 14018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:25.884097 14018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "7deb2f1433bb4232850969d5e36134b2"
format_stamp: "Formatted at 2026-08-09 04:57:25 on dist-test-slave-21b9"
server_key: "0a45ba89e7c9b3bc68eddc4730fec9e7"
server_key_iv: "5d24bdd00405216362b3bc4866e26852"
server_key_version: "encryptionkey@0"
I20260809 04:57:25.885001 14018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "7deb2f1433bb4232850969d5e36134b2"
format_stamp: "Formatted at 2026-08-09 04:57:25 on dist-test-slave-21b9"
server_key: "0a45ba89e7c9b3bc68eddc4730fec9e7"
server_key_iv: "5d24bdd00405216362b3bc4866e26852"
server_key_version: "encryptionkey@0"
I20260809 04:57:25.890112 14018 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 04:57:25.893483 14038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:25.894773 14018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:57:25.894924 14018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "7deb2f1433bb4232850969d5e36134b2"
format_stamp: "Formatted at 2026-08-09 04:57:25 on dist-test-slave-21b9"
server_key: "0a45ba89e7c9b3bc68eddc4730fec9e7"
server_key_iv: "5d24bdd00405216362b3bc4866e26852"
server_key_version: "encryptionkey@0"
I20260809 04:57:25.895114 14018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:25.901049 14018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:25.901798 14018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:25.902040 14018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:25.902880 14018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:25.904351 14018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:25.904438 14018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:25.904552 14018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:25.904625 14018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:25.934993 14018 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:35149
I20260809 04:57:25.935060 14209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:35149 every 8 connection(s)
I20260809 04:57:25.936321 14018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:57:25.937853 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 14018
I20260809 04:57:25.938017 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:57:25.938808 30671 external_mini_cluster.cc:1428] Setting key 206f90a3cde3999642c7f66d1ad4e3cd
I20260809 04:57:25.941854 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36661
--builtin_ntp_servers=127.29.243.212:38357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 04:57:25.944748 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:25.938933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:36171 (local address 127.29.243.254:36661)
0809 04:57:25.939508 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:25.939514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:25.940769 (+ 1255us) server_negotiation.cc:407] Connection header received
0809 04:57:25.942540 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:25.942545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:25.942609 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:25.942703 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:25.943992 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:25.943998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:25.944003 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:25.944023 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:25.944065 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:25.944077 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:25.944187 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:25.944370 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 04:57:25.944440 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":225,"threads_started":1}
I20260809 04:57:25.945647 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:25.939643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:36661 (local address 127.29.243.193:36171)
0809 04:57:25.940614 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:25.940661 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:25.942287 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:25.942955 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:25.942963 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:25.943051 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:25.943707 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:57:25.943709 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:25.943745 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:57:25.943827 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:25.944167 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:25.944178 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:25.944186 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:57:25.944494 (+ 308us) client_negotiation.cc:240] Negotiation successful
0809 04:57:25.944741 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":299,"threads_started":1}
I20260809 04:57:25.947016 14210 heartbeater.cc:347] Connected to a master server at 127.29.243.254:36661
I20260809 04:57:25.947355 14210 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:25.948012 14210 heartbeater.cc:511] Master 127.29.243.254:36661 requested a full tablet report, sending...
I20260809 04:57:25.950325 13909 ts_manager.cc:194] Registered new tserver with Master: 7deb2f1433bb4232850969d5e36134b2 (127.29.243.193:35149)
W20260809 04:57:26.284849 14215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:26.285116 14215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:26.285202 14215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:26.292307 14215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:26.292407 14215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:26.292514 14215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:57:26.301599 14215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:26.302883 14215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:26.304608 14215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:26.317624 14224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:26.317920 14223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:26.318089 14227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:26.318980 14215 server_base.cc:1034] running on GCE node
I20260809 04:57:26.319615 14215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:26.320544 14215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:26.321759 14215 hybrid_clock.cc:648] HybridClock initialized: now 1786251446321695 us; error 55 us; skew 500 ppm
I20260809 04:57:26.322079 14215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:57:26.324218 14215 webserver.cc:460] Webserver started at http://127.29.243.194:33697/ using document root <none> and password file <none>
I20260809 04:57:26.324872 14215 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:26.324997 14215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:26.325309 14215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:26.328126 14215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "60ecc9d8ccbc4d39a29aa0f934d4a22b"
format_stamp: "Formatted at 2026-08-09 04:57:26 on dist-test-slave-21b9"
server_key: "76ba369d35a0560136db0972d59129e7"
server_key_iv: "a146dcd5f89f5a5ceb59a6e3d5fc3d5e"
server_key_version: "encryptionkey@0"
I20260809 04:57:26.328869 14215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "60ecc9d8ccbc4d39a29aa0f934d4a22b"
format_stamp: "Formatted at 2026-08-09 04:57:26 on dist-test-slave-21b9"
server_key: "76ba369d35a0560136db0972d59129e7"
server_key_iv: "a146dcd5f89f5a5ceb59a6e3d5fc3d5e"
server_key_version: "encryptionkey@0"
I20260809 04:57:26.334129 14215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:57:26.337558 14240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:26.338894 14215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:57:26.339053 14215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "60ecc9d8ccbc4d39a29aa0f934d4a22b"
format_stamp: "Formatted at 2026-08-09 04:57:26 on dist-test-slave-21b9"
server_key: "76ba369d35a0560136db0972d59129e7"
server_key_iv: "a146dcd5f89f5a5ceb59a6e3d5fc3d5e"
server_key_version: "encryptionkey@0"
I20260809 04:57:26.339253 14215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:26.344666 14215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:26.345402 14215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:26.345638 14215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:26.346484 14215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:26.347848 14215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:26.347944 14215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:26.348050 14215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:26.348119 14215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:26.378744 14215 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:35649
I20260809 04:57:26.378790 14411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:35649 every 8 connection(s)
I20260809 04:57:26.380185 14215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:57:26.388528 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:26.382872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:52535 (local address 127.29.243.254:36661)
0809 04:57:26.383094 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:26.383099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:26.384571 (+ 1472us) server_negotiation.cc:407] Connection header received
0809 04:57:26.386279 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:26.386285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:26.386351 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:26.386451 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:26.387789 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:26.387796 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:26.387800 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:26.387817 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:26.387850 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:26.387861 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:26.387966 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:26.388160 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 04:57:26.388232 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:57:26.389428 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:26.383434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:36661 (local address 127.29.243.194:52535)
0809 04:57:26.384395 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:26.384439 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:26.386012 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:26.386681 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:26.386689 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:26.386781 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:26.387440 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:57:26.387443 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:26.387493 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:57:26.387613 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:26.387955 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:26.387965 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:26.387974 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:57:26.388250 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 04:57:26.388505 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":291,"threads_started":1}
I20260809 04:57:26.390205 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 14215
I20260809 04:57:26.390390 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:57:26.390853 14412 heartbeater.cc:347] Connected to a master server at 127.29.243.254:36661
I20260809 04:57:26.390892 30671 external_mini_cluster.cc:1428] Setting key 5c901cb71f8a7c2b1cf12358ffbb03cd
I20260809 04:57:26.391185 14412 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:26.391917 14412 heartbeater.cc:511] Master 127.29.243.254:36661 requested a full tablet report, sending...
I20260809 04:57:26.393332 13909 ts_manager.cc:194] Registered new tserver with Master: 60ecc9d8ccbc4d39a29aa0f934d4a22b (127.29.243.194:35649)
I20260809 04:57:26.393868 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36661
--builtin_ntp_servers=127.29.243.212:38357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 04:57:26.731181 14417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:26.731503 14417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:26.731635 14417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:26.738811 14417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:26.738929 14417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:26.739058 14417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:57:26.748041 14417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:38357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:36661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:26.749364 14417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:26.751066 14417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:26.763968 14428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:26.763890 14427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:26.763976 14431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:26.764660 14417 server_base.cc:1034] running on GCE node
I20260809 04:57:26.765209 14417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:26.766098 14417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:26.767362 14417 hybrid_clock.cc:648] HybridClock initialized: now 1786251446767286 us; error 50 us; skew 500 ppm
I20260809 04:57:26.767691 14417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:57:26.769920 14417 webserver.cc:460] Webserver started at http://127.29.243.195:37477/ using document root <none> and password file <none>
I20260809 04:57:26.770699 14417 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:26.770807 14417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:26.771134 14417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:26.774008 14417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "3b27dc489eb04fd2a48e6bbc315beec9"
format_stamp: "Formatted at 2026-08-09 04:57:26 on dist-test-slave-21b9"
server_key: "a4ed465abedca90df6d501d0ee1d0120"
server_key_iv: "44393a82a23aaf3e4e1ac3aa09fd6eba"
server_key_version: "encryptionkey@0"
I20260809 04:57:26.774804 14417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "3b27dc489eb04fd2a48e6bbc315beec9"
format_stamp: "Formatted at 2026-08-09 04:57:26 on dist-test-slave-21b9"
server_key: "a4ed465abedca90df6d501d0ee1d0120"
server_key_iv: "44393a82a23aaf3e4e1ac3aa09fd6eba"
server_key_version: "encryptionkey@0"
I20260809 04:57:26.779973 14417 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 04:57:26.783512 14441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:26.784860 14417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:57:26.785022 14417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "3b27dc489eb04fd2a48e6bbc315beec9"
format_stamp: "Formatted at 2026-08-09 04:57:26 on dist-test-slave-21b9"
server_key: "a4ed465abedca90df6d501d0ee1d0120"
server_key_iv: "44393a82a23aaf3e4e1ac3aa09fd6eba"
server_key_version: "encryptionkey@0"
I20260809 04:57:26.785224 14417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:26.790700 14417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:26.791493 14417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:26.791757 14417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:26.792891 14417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:26.794988 14417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:26.795083 14417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:26.795166 14417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:26.795217 14417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:26.824358 14417 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:45481
I20260809 04:57:26.824409 14617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:45481 every 8 connection(s)
I20260809 04:57:26.825683 14417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:57:26.830682 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 14417
I20260809 04:57:26.830850 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:57:26.831463 30671 external_mini_cluster.cc:1428] Setting key 8ec76c7094f68327dcff2bfac4372b0a
I20260809 04:57:26.834380 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:26.828324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:34871 (local address 127.29.243.254:36661)
0809 04:57:26.828529 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:26.828533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:26.829883 (+ 1350us) server_negotiation.cc:407] Connection header received
0809 04:57:26.832189 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:26.832195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:26.832266 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:26.832377 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:26.833719 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:26.833725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:26.833729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:26.833746 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:26.833778 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:26.833790 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:26.833879 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:26.834095 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 04:57:26.834144 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 04:57:26.835557 14621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:26.828827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:36661 (local address 127.29.243.195:34871)
0809 04:57:26.829739 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:26.829775 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:26.831762 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:26.832638 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:26.832649 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:26.832760 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:26.833429 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:57:26.833432 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:26.833477 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:57:26.833563 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:26.833886 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:26.833897 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:26.833905 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:57:26.834174 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 04:57:26.834431 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":263,"threads_started":1}
I20260809 04:57:26.836931 14618 heartbeater.cc:347] Connected to a master server at 127.29.243.254:36661
I20260809 04:57:26.837239 14618 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:26.837826 14618 heartbeater.cc:511] Master 127.29.243.254:36661 requested a full tablet report, sending...
I20260809 04:57:26.839205 13909 ts_manager.cc:194] Registered new tserver with Master: 3b27dc489eb04fd2a48e6bbc315beec9 (127.29.243.195:45481)
I20260809 04:57:26.847010 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:57:26.853080 14212 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:57:26.851463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.29.243.254:36661)
0809 04:57:26.851629 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:26.851634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:26.852125 (+ 491us) server_negotiation.cc:407] Connection header received
0809 04:57:26.852282 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:26.852288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:26.852348 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:26.852430 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:26.852822 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41108: BlockingRecv error: recv got EOF from 127.0.0.1:41108 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:57:26.853627 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 14018
I20260809 04:57:26.856869 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:26.854498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57844 (local address 127.29.243.193:35149)
0809 04:57:26.855004 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:26.855011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:26.855038 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:57:26.855096 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:26.855102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:26.855366 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:26.855487 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:26.856323 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:26.856330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:26.856341 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:26.856353 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:26.856397 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:26.856415 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:26.856521 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:26.856566 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 04:57:26.856651 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":215,"threads_started":1}
I20260809 04:57:26.858201 14202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:26.938863 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 14018
I20260809 04:57:26.966324 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 14215
I20260809 04:57:26.969863 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:26.967228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.29.243.194:35649)
0809 04:57:26.967847 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:26.967855 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:26.967885 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:57:26.967957 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:26.967964 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:26.968200 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:26.968343 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:26.969221 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:26.969229 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:26.969245 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:26.969258 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:26.969305 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:26.969320 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:26.969468 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:26.969520 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 04:57:26.969616 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":288,"threads_started":1}
I20260809 04:57:26.971179 14405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:27.050944 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 14215
I20260809 04:57:27.075888 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 14417
I20260809 04:57:27.079991 14638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:27.077085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.29.243.195:45481)
0809 04:57:27.077784 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:27.077791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:27.077823 (+ 32us) server_negotiation.cc:407] Connection header received
0809 04:57:27.077891 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:27.077898 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:27.078134 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:27.078262 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:27.079186 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:27.079193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:27.079205 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:57:27.079228 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:27.079289 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:57:27.079309 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:27.079459 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:27.079556 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 04:57:27.079653 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":379,"threads_started":1}
I20260809 04:57:27.081383 14611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:27.162720 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 14417
I20260809 04:57:27.187707 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 13865
I20260809 04:57:27.189023 13967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:27.291859 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 13865
2026-08-09T04:57:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2455 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:57:27 dist-test-slave-21b9 krb5kdc[14645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:57:27 dist-test-slave-21b9 krb5kdc[14645](info): set up 2 sockets
Aug 09 04:57:27 dist-test-slave-21b9 krb5kdc[14645](info): commencing operation
krb5kdc: starting...
W20260809 04:57:28.569455 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:57:28 dist-test-slave-21b9 krb5kdc[14645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:57:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:57:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:57:28.844509 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:32789
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:32789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:57:29.186720 14665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:29.187033 14665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:29.187170 14665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:29.194437 14665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:57:29.194566 14665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:29.194651 14665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:29.194728 14665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:57:29.194792 14665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:57:29.204712 14665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:32789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:32789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:29.206073 14665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:29.207381 14665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:29.218792 14674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:29.218890 14671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:29.218865 14681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:29.220036 14665 server_base.cc:1034] running on GCE node
I20260809 04:57:29.220777 14665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:29.222146 14665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:29.223357 14665 hybrid_clock.cc:648] HybridClock initialized: now 1786251449223308 us; error 33 us; skew 500 ppm
I20260809 04:57:29.223690 14665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:29 dist-test-slave-21b9 krb5kdc[14645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251449, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:29.227030 14665 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:57:29.228706 14665 webserver.cc:460] Webserver started at http://127.29.243.254:34157/ using document root <none> and password file <none>
I20260809 04:57:29.229416 14665 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:29.229528 14665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:29.229846 14665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:29.232877 14665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "92d85109c4c34259bfda6b17572883c6"
format_stamp: "Formatted at 2026-08-09 04:57:29 on dist-test-slave-21b9"
server_key: "28abdf4ccc49fa3516576d1023b298b7"
server_key_iv: "18a3c2be508066a671588778710cbbbf"
server_key_version: "encryptionkey@0"
I20260809 04:57:29.233634 14665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "92d85109c4c34259bfda6b17572883c6"
format_stamp: "Formatted at 2026-08-09 04:57:29 on dist-test-slave-21b9"
server_key: "28abdf4ccc49fa3516576d1023b298b7"
server_key_iv: "18a3c2be508066a671588778710cbbbf"
server_key_version: "encryptionkey@0"
I20260809 04:57:29.238898 14665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 04:57:29.242339 14690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:29.243790 14665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:57:29.243960 14665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "92d85109c4c34259bfda6b17572883c6"
format_stamp: "Formatted at 2026-08-09 04:57:29 on dist-test-slave-21b9"
server_key: "28abdf4ccc49fa3516576d1023b298b7"
server_key_iv: "18a3c2be508066a671588778710cbbbf"
server_key_version: "encryptionkey@0"
I20260809 04:57:29.244145 14665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:29.266937 14665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:29.271253 14665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:29.271551 14665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:29.289702 14665 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:32789
I20260809 04:57:29.289734 14769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:32789 every 8 connection(s)
I20260809 04:57:29.291051 14665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:57:29.291736 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 14665
I20260809 04:57:29.291952 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:57:29.292838 30671 external_mini_cluster.cc:1428] Setting key 0281f566e663d01f3c7d473a0998b29d
I20260809 04:57:29.295892 14771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:57:29 dist-test-slave-21b9 krb5kdc[14645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:57:29.309057 14771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6: Bootstrap starting.
I20260809 04:57:29.312765 14771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6: Neither blocks nor log segments found. Creating new log.
I20260809 04:57:29.313941 14771 log.cc:824] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6: Log is configured to *not* fsync() on all Append() calls
I20260809 04:57:29.316057 14779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:29.294905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43070 (local address 127.29.243.254:32789)
0809 04:57:29.295655 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:29.295668 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:29.295710 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:57:29.297330 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:29.297364 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:29.297750 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:29.298067 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:29.299959 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:29.301284 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:29.303105 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:29.303834 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:29.303969 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:29.308683 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:29.308691 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:29.308706 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:29.308742 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:29.311800 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:29.312587 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:29.312593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:29.312600 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:29.312768 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:29.313190 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:29.313194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:29.313196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:29.313733 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:29.313964 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:29.314414 (+ 450us) server_negotiation.cc:299] Negotiation successful
0809 04:57:29.314874 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":330,"threads_started":1}
I20260809 04:57:29.317055 14771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6: No bootstrap required, opened a new log
I20260809 04:57:29.324888 14771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d85109c4c34259bfda6b17572883c6" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32789 } }
I20260809 04:57:29.325245 14771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:57:29.325333 14771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d85109c4c34259bfda6b17572883c6, State: Initialized, Role: FOLLOWER
I20260809 04:57:29.325863 14771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d85109c4c34259bfda6b17572883c6" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32789 } }
I20260809 04:57:29.326092 14771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:57:29.326218 14771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:57:29.326375 14771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:57:29.328557 14771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d85109c4c34259bfda6b17572883c6" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32789 } }
I20260809 04:57:29.329031 14771 leader_election.cc:302] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92d85109c4c34259bfda6b17572883c6; no voters:
I20260809 04:57:29.329617 14771 leader_election.cc:288] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:57:29.329746 14783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:57:29.330372 14783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [term 1 LEADER]: Becoming Leader. State: Replica: 92d85109c4c34259bfda6b17572883c6, State: Running, Role: LEADER
I20260809 04:57:29.330892 14783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d85109c4c34259bfda6b17572883c6" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32789 } }
I20260809 04:57:29.331679 14771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:57:29.333777 14784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92d85109c4c34259bfda6b17572883c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d85109c4c34259bfda6b17572883c6" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32789 } } }
I20260809 04:57:29.333897 14785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92d85109c4c34259bfda6b17572883c6. Latest consensus state: current_term: 1 leader_uuid: "92d85109c4c34259bfda6b17572883c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d85109c4c34259bfda6b17572883c6" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 32789 } } }
I20260809 04:57:29.334172 14784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [sys.catalog]: This master's current role is: LEADER
I20260809 04:57:29.334239 14785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6 [sys.catalog]: This master's current role is: LEADER
I20260809 04:57:29.335953 14793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:57:29.341383 14793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:57:29.349740 14793 catalog_manager.cc:1347] Generated new cluster ID: be82ea3b0f6f466b8f41f58bc3fca51a
I20260809 04:57:29.349844 14793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:57:29.378367 14793 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:57:29.379806 14793 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:57:29.388029 14793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6: Generated new TSK 0
I20260809 04:57:29.388880 14793 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:57:29.470520 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32789
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:57:29.811116 14811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:29.811437 14811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:29.811584 14811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:29.819093 14811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:29.819198 14811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:29.819356 14811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:57:29.829397 14811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:29.830816 14811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:29.832254 14811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:29.845875 14820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:29.845906 14821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:29.846057 14824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:29.846602 14811 server_base.cc:1034] running on GCE node
I20260809 04:57:29.847230 14811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:29.848241 14811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:29.849453 14811 hybrid_clock.cc:648] HybridClock initialized: now 1786251449849405 us; error 64 us; skew 500 ppm
I20260809 04:57:29.849757 14811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:29 dist-test-slave-21b9 krb5kdc[14645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251449, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:29.852979 14811 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:57:29.854429 14811 webserver.cc:460] Webserver started at http://127.29.243.193:33959/ using document root <none> and password file <none>
I20260809 04:57:29.855123 14811 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:29.855245 14811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:29.855582 14811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:29.858537 14811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "cda8a6bcca62485683954060cfbf594d"
format_stamp: "Formatted at 2026-08-09 04:57:29 on dist-test-slave-21b9"
server_key: "45c8a3deedc7aa81c0f40c07ff408923"
server_key_iv: "8e47906ee8cc63d0afec5c571e89cdf9"
server_key_version: "encryptionkey@0"
I20260809 04:57:29.859599 14811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "cda8a6bcca62485683954060cfbf594d"
format_stamp: "Formatted at 2026-08-09 04:57:29 on dist-test-slave-21b9"
server_key: "45c8a3deedc7aa81c0f40c07ff408923"
server_key_iv: "8e47906ee8cc63d0afec5c571e89cdf9"
server_key_version: "encryptionkey@0"
I20260809 04:57:29.864724 14811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 04:57:29.868268 14835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:29.869668 14811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:57:29.869830 14811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "cda8a6bcca62485683954060cfbf594d"
format_stamp: "Formatted at 2026-08-09 04:57:29 on dist-test-slave-21b9"
server_key: "45c8a3deedc7aa81c0f40c07ff408923"
server_key_iv: "8e47906ee8cc63d0afec5c571e89cdf9"
server_key_version: "encryptionkey@0"
I20260809 04:57:29.870025 14811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:29.893807 14811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:29.897850 14811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:29.898130 14811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:29.899093 14811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:29.900494 14811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:29.900575 14811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:29.900695 14811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:29.900749 14811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:29.930689 14811 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:43601
I20260809 04:57:29.930748 15003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:43601 every 8 connection(s)
I20260809 04:57:29.932101 14811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:57:29.938799 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 14811
I20260809 04:57:29.938998 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:57:29.939710 30671 external_mini_cluster.cc:1428] Setting key 6fe289f4c7ed80abeade262dd56aa309
Aug 09 04:57:29 dist-test-slave-21b9 krb5kdc[14645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251449, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:57:29.953828 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:29.934856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:58325 (local address 127.29.243.254:32789)
0809 04:57:29.935371 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:29.935375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:29.936230 (+ 855us) server_negotiation.cc:407] Connection header received
0809 04:57:29.938348 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:29.938355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:29.938456 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:29.938564 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:29.940287 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:29.941178 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:29.942868 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:29.943568 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:29.943733 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:29.947224 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:29.947231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:29.947236 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:29.947334 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:29.950498 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:29.951258 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:29.951265 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:29.951301 (+ 36us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:29.951468 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:29.951952 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:29.951957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:29.951960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:29.952343 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:29.952489 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:29.953197 (+ 708us) server_negotiation.cc:299] Negotiation successful
0809 04:57:29.953437 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":236,"threads_started":1}
I20260809 04:57:29.954569 15009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:29.935291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:32789 (local address 127.29.243.193:58325)
0809 04:57:29.936052 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:29.936100 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:29.938052 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:29.938824 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:29.938833 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:29.939465 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:29.940098 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:29.940119 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:29.941369 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:29.941374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:29.942696 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:29.942710 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:29.943797 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:29.943803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:29.943998 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:29.944658 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:29.944680 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:29.946982 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:29.950673 (+ 3691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:29.950680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:29.950693 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:29.951133 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:29.951621 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:29.951625 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:29.951628 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:29.951835 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:29.952475 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:29.952481 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:29.952802 (+ 321us) client_negotiation.cc:769] Sending connection context
0809 04:57:29.953054 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 04:57:29.953463 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":273,"threads_started":1}
I20260809 04:57:29.956216 15004 heartbeater.cc:347] Connected to a master server at 127.29.243.254:32789
I20260809 04:57:29.956573 15004 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:29.957370 15004 heartbeater.cc:511] Master 127.29.243.254:32789 requested a full tablet report, sending...
I20260809 04:57:29.959859 14715 ts_manager.cc:194] Registered new tserver with Master: cda8a6bcca62485683954060cfbf594d (127.29.243.193:43601)
I20260809 04:57:29.961843 14715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:58325
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:57:30.007151 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32789
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:57:30.350068 15015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:30.350334 15015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:30.350426 15015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:30.357491 15015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:30.357594 15015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:30.357707 15015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:57:30.367057 15015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:30.368453 15015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:30.369827 15015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:30.383018 15025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:30.383000 15027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:30.383363 15032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:30.383787 15015 server_base.cc:1034] running on GCE node
I20260809 04:57:30.384387 15015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:30.385360 15015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:30.386574 15015 hybrid_clock.cc:648] HybridClock initialized: now 1786251450386522 us; error 45 us; skew 500 ppm
I20260809 04:57:30.386897 15015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:30 dist-test-slave-21b9 krb5kdc[14645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251450, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:30.390129 15015 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:57:30.391644 15015 webserver.cc:460] Webserver started at http://127.29.243.194:33961/ using document root <none> and password file <none>
I20260809 04:57:30.392321 15015 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:30.392452 15015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:30.392747 15015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:30.395793 15015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "d6c8a069da964e028f29b12db7ee1167"
format_stamp: "Formatted at 2026-08-09 04:57:30 on dist-test-slave-21b9"
server_key: "9d25d3325ebd3f960f1720850f468764"
server_key_iv: "9567f4151525f878448c25f1f5066658"
server_key_version: "encryptionkey@0"
I20260809 04:57:30.396577 15015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "d6c8a069da964e028f29b12db7ee1167"
format_stamp: "Formatted at 2026-08-09 04:57:30 on dist-test-slave-21b9"
server_key: "9d25d3325ebd3f960f1720850f468764"
server_key_iv: "9567f4151525f878448c25f1f5066658"
server_key_version: "encryptionkey@0"
I20260809 04:57:30.401734 15015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:57:30.405133 15042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:30.406378 15015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:57:30.406544 15015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "d6c8a069da964e028f29b12db7ee1167"
format_stamp: "Formatted at 2026-08-09 04:57:30 on dist-test-slave-21b9"
server_key: "9d25d3325ebd3f960f1720850f468764"
server_key_iv: "9567f4151525f878448c25f1f5066658"
server_key_version: "encryptionkey@0"
I20260809 04:57:30.406714 15015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:30.428798 15015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:30.433071 15015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:30.433337 15015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:30.434293 15015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:30.435662 15015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:30.435746 15015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:30.435861 15015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:30.435915 15015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:30.468433 15015 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:46557
I20260809 04:57:30.468466 15205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:46557 every 8 connection(s)
I20260809 04:57:30.469862 15015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:57:30.476308 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 15015
I20260809 04:57:30.476480 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:57:30.476915 30671 external_mini_cluster.cc:1428] Setting key b70ff918749715bc253d0aaf256cad4e
Aug 09 04:57:30 dist-test-slave-21b9 krb5kdc[14645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251450, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:57:30.489356 15212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:30.472599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:38575 (local address 127.29.243.254:32789)
0809 04:57:30.473057 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:30.473062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:30.474133 (+ 1071us) server_negotiation.cc:407] Connection header received
0809 04:57:30.476019 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:30.476024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:30.476121 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:30.476221 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:30.477761 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:30.478538 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:30.480093 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:30.480525 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:30.480635 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:30.483959 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:30.483964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:30.483967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:30.483999 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:30.486360 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:30.487035 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:30.487039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:30.487041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:30.487170 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:30.487604 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:30.487607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:30.487609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:30.487932 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:30.488081 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:30.488754 (+ 673us) server_negotiation.cc:299] Negotiation successful
0809 04:57:30.488981 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":228,"threads_started":1}
I20260809 04:57:30.490409 15214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:30.473190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:32789 (local address 127.29.243.194:38575)
0809 04:57:30.473964 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:30.474021 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:30.475748 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:30.476476 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:30.476486 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:30.477019 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:30.477587 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:30.477605 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:30.478688 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:30.478692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:30.479948 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:30.479960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:30.480742 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:30.480750 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:30.480970 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:30.481624 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:30.481646 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:30.483746 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:30.486488 (+ 2742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:30.486497 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:30.486513 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:30.486910 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:30.487291 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:30.487296 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:30.487300 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:30.487486 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:30.488073 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:30.488080 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:30.488398 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 04:57:30.488633 (+ 235us) client_negotiation.cc:240] Negotiation successful
0809 04:57:30.489164 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":332,"threads_started":1}
I20260809 04:57:30.491882 15206 heartbeater.cc:347] Connected to a master server at 127.29.243.254:32789
I20260809 04:57:30.492213 15206 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:30.493080 15206 heartbeater.cc:511] Master 127.29.243.254:32789 requested a full tablet report, sending...
I20260809 04:57:30.494717 14715 ts_manager.cc:194] Registered new tserver with Master: d6c8a069da964e028f29b12db7ee1167 (127.29.243.194:46557)
I20260809 04:57:30.495570 14715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:38575
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:57:30.538288 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32789
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:57:30.875026 15218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:57:30.875324 15218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:57:30.875468 15218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:57:30.883015 15218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:57:30.883121 15218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:57:30.883245 15218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:57:30.892887 15218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:32789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:57:30.894124 15218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:57:30.895462 15218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:57:30.908221 15224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:30.908164 15228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:57:30.908200 15226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:57:30.908913 15218 server_base.cc:1034] running on GCE node
I20260809 04:57:30.909480 15218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:57:30.910449 15218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:57:30.911649 15218 hybrid_clock.cc:648] HybridClock initialized: now 1786251450911594 us; error 43 us; skew 500 ppm
I20260809 04:57:30.911954 15218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:57:30 dist-test-slave-21b9 krb5kdc[14645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251450, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:57:30.915021 15218 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:57:30.916501 15218 webserver.cc:460] Webserver started at http://127.29.243.195:37765/ using document root <none> and password file <none>
I20260809 04:57:30.917169 15218 fs_manager.cc:359] Metadata directory not provided
I20260809 04:57:30.917290 15218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:57:30.917600 15218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:57:30.920554 15218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "f2b42f47799d46eaaace2658e186b070"
format_stamp: "Formatted at 2026-08-09 04:57:30 on dist-test-slave-21b9"
server_key: "7758ebc363a18869310e506b96bbbd77"
server_key_iv: "bbdb393d18a0712c033394e7af306746"
server_key_version: "encryptionkey@0"
I20260809 04:57:30.921314 15218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "f2b42f47799d46eaaace2658e186b070"
format_stamp: "Formatted at 2026-08-09 04:57:30 on dist-test-slave-21b9"
server_key: "7758ebc363a18869310e506b96bbbd77"
server_key_iv: "bbdb393d18a0712c033394e7af306746"
server_key_version: "encryptionkey@0"
I20260809 04:57:30.926522 15218 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:57:30.930019 15237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:30.931540 15218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:57:30.931692 15218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "f2b42f47799d46eaaace2658e186b070"
format_stamp: "Formatted at 2026-08-09 04:57:30 on dist-test-slave-21b9"
server_key: "7758ebc363a18869310e506b96bbbd77"
server_key_iv: "bbdb393d18a0712c033394e7af306746"
server_key_version: "encryptionkey@0"
I20260809 04:57:30.931968 15218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:57:30.948117 15218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:57:30.952162 15218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:57:30.952461 15218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:57:30.953464 15218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:57:30.954891 15218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:57:30.954973 15218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:30.955107 15218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:57:30.955171 15218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:57:30.965200 15004 heartbeater.cc:503] Master 127.29.243.254:32789 was elected leader, sending a full tablet report...
I20260809 04:57:30.986299 15218 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:46777
I20260809 04:57:30.986332 15399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:46777 every 8 connection(s)
I20260809 04:57:30.987701 15218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:57:30.994017 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 15218
I20260809 04:57:30.994184 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:57:30.994606 30671 external_mini_cluster.cc:1428] Setting key 5d72c1e9498ba2431b247a41bc91975d
Aug 09 04:57:30 dist-test-slave-21b9 krb5kdc[14645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251450, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:57:31.006920 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:30.990302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:49949 (local address 127.29.243.254:32789)
0809 04:57:30.990773 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:30.990778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:30.991652 (+ 874us) server_negotiation.cc:407] Connection header received
0809 04:57:30.993458 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:30.993465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:30.993566 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:30.993676 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:30.995298 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:30.996230 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:30.998061 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:30.998454 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:30.998557 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.001836 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:31.001841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:31.001844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:31.001879 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:31.004439 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:31.005072 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:31.005075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:31.005078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:31.005193 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:31.005551 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:31.005553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:31.005555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:31.005801 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:31.005917 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:31.006490 (+ 573us) server_negotiation.cc:299] Negotiation successful
0809 04:57:31.006653 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":237,"threads_started":1}
I20260809 04:57:31.008056 15406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:30.990757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:32789 (local address 127.29.243.195:49949)
0809 04:57:30.991494 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:30.991542 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:30.993186 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:30.993889 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:30.993897 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:30.994485 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:57:30.995093 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:30.995112 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:30.996381 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:30.996385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:30.997930 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:30.997946 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:30.998637 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:30.998642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:30.998834 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:30.999475 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:57:30.999497 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:57:31.001630 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:57:31.004564 (+ 2934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:31.004571 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:31.004583 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:31.004981 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:31.005267 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:57:31.005270 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:57:31.005272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:57:31.005474 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:57:31.005890 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:57:31.005895 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:57:31.006200 (+ 305us) client_negotiation.cc:769] Sending connection context
0809 04:57:31.006421 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 04:57:31.006832 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":281,"threads_started":1}
I20260809 04:57:31.009567 15400 heartbeater.cc:347] Connected to a master server at 127.29.243.254:32789
I20260809 04:57:31.009868 15400 heartbeater.cc:464] Registering TS with master...
I20260809 04:57:31.010713 15400 heartbeater.cc:511] Master 127.29.243.254:32789 requested a full tablet report, sending...
I20260809 04:57:31.012228 14714 ts_manager.cc:194] Registered new tserver with Master: f2b42f47799d46eaaace2658e186b070 (127.29.243.195:46777)
I20260809 04:57:31.013136 14714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:49949
I20260809 04:57:31.022390 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:57:31.039258 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.026818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.29.243.254:32789)
0809 04:57:31.026975 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:31.026979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:31.027523 (+ 544us) server_negotiation.cc:407] Connection header received
0809 04:57:31.027697 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:31.027701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:31.027781 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:31.027868 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:31.029248 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.029977 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.031256 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.031718 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.031831 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.034090 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:31.034095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:31.034098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:31.034130 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:31.036634 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:31.037275 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:31.037279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:31.037281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:31.037399 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:31.037792 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:31.037798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:31.037802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:31.038141 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:31.038376 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:31.038768 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 04:57:31.038997 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:57:31.043205 14714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:57:31.050037 14714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:57:31.089627 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.063329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39608 (local address 127.29.243.195:46777)
0809 04:57:31.074378 (+ 11049us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:31.074390 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:31.080784 (+ 6394us) server_negotiation.cc:407] Connection header received
0809 04:57:31.081138 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:31.081146 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:31.081410 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:31.081567 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:57:31.082843 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.084670 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.086947 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.088297 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.088494 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.088676 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:31.089075 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 04:57:31.089255 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10244,"thread_start_us":10071,"threads_started":1}
I20260809 04:57:31.090045 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.063636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46524 (local address 127.29.243.194:46557)
0809 04:57:31.080124 (+ 16488us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:31.080135 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:31.081296 (+ 1161us) server_negotiation.cc:407] Connection header received
0809 04:57:31.081404 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:31.081412 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:31.081650 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:31.081836 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:57:31.083151 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.084245 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.087529 (+ 3284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.088458 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.088631 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.088805 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:31.089568 (+ 763us) server_negotiation.cc:299] Negotiation successful
0809 04:57:31.089731 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16285,"thread_start_us":16173,"threads_started":1}
I20260809 04:57:31.090436 15421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.062995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48434 (local address 127.29.243.193:43601)
0809 04:57:31.081433 (+ 18438us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:31.081442 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:31.081467 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:57:31.081558 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:31.081563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:31.081772 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:31.081957 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:57:31.083064 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.084637 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.087336 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.088728 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.089104 (+ 376us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.089273 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:31.089479 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 04:57:31.089637 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18219,"thread_start_us":18040,"threads_started":1}
I20260809 04:57:31.090701 15418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.062572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43601 (local address 127.0.0.1:48434)
0809 04:57:31.078292 (+ 15720us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:31.078333 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:31.078550 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:31.082173 (+ 3623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:31.082180 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:31.082200 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:57:31.082819 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:31.082833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.084783 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.084788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:31.087148 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:31.087163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.089010 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.089019 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:31.089245 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.089321 (+ 76us) client_negotiation.cc:769] Sending connection context
0809 04:57:31.089404 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 04:57:31.089611 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15457,"mutex_wait_us":71,"thread_start_us":15295,"threads_started":1}
I20260809 04:57:31.090844 15419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.063098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:46557 (local address 127.0.0.1:46524)
0809 04:57:31.081003 (+ 17905us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:31.081023 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:31.081163 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:31.082131 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:31.082135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:31.082163 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:57:31.082959 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:31.082973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.084387 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.084393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:31.087377 (+ 2984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:31.087388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.089075 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.089085 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:31.089318 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.089352 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 04:57:31.089439 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 04:57:31.090391 (+ 952us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17760,"mutex_wait_us":963,"spinlock_wait_cycles":5504,"thread_start_us":17194,"threads_started":1}
I20260809 04:57:31.091349 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.062691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:46777 (local address 127.0.0.1:39608)
0809 04:57:31.080632 (+ 17941us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:31.080653 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:31.080923 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:31.081915 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:31.081922 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:31.081954 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:57:31.082566 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:31.082583 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.084823 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.084827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:31.086778 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:31.086793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.088498 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.088505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:31.088701 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.088752 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 04:57:31.088900 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 04:57:31.090041 (+ 1141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17781,"mutex_wait_us":4,"thread_start_us":18019,"threads_started":1}
I20260809 04:57:31.094660 15309 tablet_service.cc:1467] Processing CreateTablet for tablet 68de036c16304d23a7a305e2a31cfee5 (DEFAULT_TABLE table=test-table [id=838e4052804b41fab7e7b700c3dec8da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:57:31.094800 14909 tablet_service.cc:1467] Processing CreateTablet for tablet 68de036c16304d23a7a305e2a31cfee5 (DEFAULT_TABLE table=test-table [id=838e4052804b41fab7e7b700c3dec8da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:57:31.094797 15115 tablet_service.cc:1467] Processing CreateTablet for tablet 68de036c16304d23a7a305e2a31cfee5 (DEFAULT_TABLE table=test-table [id=838e4052804b41fab7e7b700c3dec8da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:57:31.096262 14909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68de036c16304d23a7a305e2a31cfee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:57:31.096263 15115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68de036c16304d23a7a305e2a31cfee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:57:31.096271 15309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68de036c16304d23a7a305e2a31cfee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:57:31.107841 15426 tablet_bootstrap.cc:492] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: Bootstrap starting.
I20260809 04:57:31.111205 15428 tablet_bootstrap.cc:492] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: Bootstrap starting.
I20260809 04:57:31.113307 15426 tablet_bootstrap.cc:654] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: Neither blocks nor log segments found. Creating new log.
I20260809 04:57:31.114657 15426 log.cc:824] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: Log is configured to *not* fsync() on all Append() calls
I20260809 04:57:31.114871 15428 tablet_bootstrap.cc:654] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: Neither blocks nor log segments found. Creating new log.
I20260809 04:57:31.114984 15430 tablet_bootstrap.cc:492] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: Bootstrap starting.
I20260809 04:57:31.115984 15428 log.cc:824] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: Log is configured to *not* fsync() on all Append() calls
I20260809 04:57:31.120956 15430 tablet_bootstrap.cc:654] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: Neither blocks nor log segments found. Creating new log.
I20260809 04:57:31.122310 15426 tablet_bootstrap.cc:492] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: No bootstrap required, opened a new log
I20260809 04:57:31.122774 15430 log.cc:824] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:57:31.122802 15426 ts_tablet_manager.cc:1397] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.002s
I20260809 04:57:31.123162 15428 tablet_bootstrap.cc:492] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: No bootstrap required, opened a new log
I20260809 04:57:31.123708 15428 ts_tablet_manager.cc:1397] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260809 04:57:31.128472 15430 tablet_bootstrap.cc:492] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: No bootstrap required, opened a new log
I20260809 04:57:31.128928 15430 ts_tablet_manager.cc:1397] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.000s
I20260809 04:57:31.149092 15430 raft_consensus.cc:348] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda8a6bcca62485683954060cfbf594d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43601 } } peers { permanent_uuid: "f2b42f47799d46eaaace2658e186b070" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 46777 } } peers { permanent_uuid: "d6c8a069da964e028f29b12db7ee1167" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46557 } }
I20260809 04:57:31.149266 15426 raft_consensus.cc:348] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda8a6bcca62485683954060cfbf594d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43601 } } peers { permanent_uuid: "f2b42f47799d46eaaace2658e186b070" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 46777 } } peers { permanent_uuid: "d6c8a069da964e028f29b12db7ee1167" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46557 } }
I20260809 04:57:31.149266 15428 raft_consensus.cc:348] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda8a6bcca62485683954060cfbf594d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43601 } } peers { permanent_uuid: "f2b42f47799d46eaaace2658e186b070" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 46777 } } peers { permanent_uuid: "d6c8a069da964e028f29b12db7ee1167" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46557 } }
I20260809 04:57:31.150383 15430 raft_consensus.cc:374] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:57:31.150473 15428 raft_consensus.cc:374] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:57:31.150473 15426 raft_consensus.cc:374] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:57:31.150635 15430 raft_consensus.cc:729] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cda8a6bcca62485683954060cfbf594d, State: Initialized, Role: FOLLOWER
I20260809 04:57:31.150653 15428 raft_consensus.cc:729] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c8a069da964e028f29b12db7ee1167, State: Initialized, Role: FOLLOWER
I20260809 04:57:31.150653 15426 raft_consensus.cc:729] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b42f47799d46eaaace2658e186b070, State: Initialized, Role: FOLLOWER
I20260809 04:57:31.151952 15430 consensus_queue.cc:260] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda8a6bcca62485683954060cfbf594d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43601 } } peers { permanent_uuid: "f2b42f47799d46eaaace2658e186b070" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 46777 } } peers { permanent_uuid: "d6c8a069da964e028f29b12db7ee1167" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46557 } }
I20260809 04:57:31.152071 15426 consensus_queue.cc:260] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda8a6bcca62485683954060cfbf594d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43601 } } peers { permanent_uuid: "f2b42f47799d46eaaace2658e186b070" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 46777 } } peers { permanent_uuid: "d6c8a069da964e028f29b12db7ee1167" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46557 } }
I20260809 04:57:31.152067 15428 consensus_queue.cc:260] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda8a6bcca62485683954060cfbf594d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43601 } } peers { permanent_uuid: "f2b42f47799d46eaaace2658e186b070" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 46777 } } peers { permanent_uuid: "d6c8a069da964e028f29b12db7ee1167" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46557 } }
I20260809 04:57:31.156808 15430 ts_tablet_manager.cc:1428] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: Time spent starting tablet: real 0.028s user 0.018s sys 0.009s
I20260809 04:57:31.159147 15400 heartbeater.cc:503] Master 127.29.243.254:32789 was elected leader, sending a full tablet report...
I20260809 04:57:31.160225 15206 heartbeater.cc:503] Master 127.29.243.254:32789 was elected leader, sending a full tablet report...
I20260809 04:57:31.161197 15426 ts_tablet_manager.cc:1428] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: Time spent starting tablet: real 0.038s user 0.027s sys 0.013s
I20260809 04:57:31.164819 15428 ts_tablet_manager.cc:1428] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: Time spent starting tablet: real 0.041s user 0.024s sys 0.012s
W20260809 04:57:31.184806 15006 tablet.cc:2369] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:57:31.222256 15211 tablet.cc:2369] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:57:31.239446 15401 tablet.cc:2369] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:57:31.252840 15438 raft_consensus.cc:482] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:57:31.253423 15438 raft_consensus.cc:504] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda8a6bcca62485683954060cfbf594d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43601 } } peers { permanent_uuid: "f2b42f47799d46eaaace2658e186b070" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 46777 } } peers { permanent_uuid: "d6c8a069da964e028f29b12db7ee1167" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46557 } }
I20260809 04:57:31.257408 15438 leader_election.cc:288] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cda8a6bcca62485683954060cfbf594d (127.29.243.193:43601), d6c8a069da964e028f29b12db7ee1167 (127.29.243.194:46557)
I20260809 04:57:31.269351 15406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.257864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43601 (local address 127.29.243.195:42643)
0809 04:57:31.258199 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:31.258230 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:31.258648 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:31.260082 (+ 1434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:31.260090 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:31.260153 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:57:31.260645 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:31.260663 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.262669 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.262675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:31.265386 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:31.265412 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.267684 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.267741 (+ 57us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:31.268283 (+ 542us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.268335 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 04:57:31.268479 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 04:57:31.268689 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":1}
I20260809 04:57:31.269320 15421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.258580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:42643 (local address 127.29.243.193:43601)
0809 04:57:31.259050 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:31.259096 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:31.259235 (+ 139us) server_negotiation.cc:407] Connection header received
0809 04:57:31.259463 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:31.259480 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:31.259660 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:31.259837 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:57:31.260891 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.262296 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.265724 (+ 3428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.267429 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.267771 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.267981 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:31.268522 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 04:57:31.268731 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260809 04:57:31.270301 15439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.258521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:46557 (local address 127.29.243.195:46611)
0809 04:57:31.259887 (+ 1366us) negotiation.cc:107] Waiting for socket to connect
0809 04:57:31.259903 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:57:31.260283 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:57:31.260990 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:57:31.261006 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:57:31.261062 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:57:31.261715 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:31.261734 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.263817 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.263824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:31.266587 (+ 2763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:57:31.266609 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.268878 (+ 2269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.268887 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:57:31.269302 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.269348 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 04:57:31.269479 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 04:57:31.269712 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1217,"spinlock_wait_cycles":122112,"thread_start_us":915,"threads_started":1}
I20260809 04:57:31.270627 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.258698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:46611 (local address 127.29.243.194:46557)
0809 04:57:31.259321 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:31.259342 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:31.260089 (+ 747us) server_negotiation.cc:407] Connection header received
0809 04:57:31.260680 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:31.260687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:31.260811 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:31.260964 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:57:31.261967 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.263568 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.266980 (+ 3412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.268599 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.268828 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.269133 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:31.269838 (+ 705us) server_negotiation.cc:299] Negotiation successful
0809 04:57:31.270143 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260809 04:57:31.271409 14937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68de036c16304d23a7a305e2a31cfee5" candidate_uuid: "f2b42f47799d46eaaace2658e186b070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cda8a6bcca62485683954060cfbf594d" is_pre_election: true
I20260809 04:57:31.272737 14937 raft_consensus.cc:2454] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b42f47799d46eaaace2658e186b070 in term 0.
I20260809 04:57:31.272418 15140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68de036c16304d23a7a305e2a31cfee5" candidate_uuid: "f2b42f47799d46eaaace2658e186b070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c8a069da964e028f29b12db7ee1167" is_pre_election: true
I20260809 04:57:31.273754 15140 raft_consensus.cc:2454] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b42f47799d46eaaace2658e186b070 in term 0.
I20260809 04:57:31.274138 15242 leader_election.cc:302] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cda8a6bcca62485683954060cfbf594d, f2b42f47799d46eaaace2658e186b070; no voters:
I20260809 04:57:31.275146 15438 raft_consensus.cc:2781] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:57:31.275413 15438 raft_consensus.cc:482] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:57:31.275593 15438 raft_consensus.cc:3037] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:57:31.279654 15438 raft_consensus.cc:504] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda8a6bcca62485683954060cfbf594d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43601 } } peers { permanent_uuid: "f2b42f47799d46eaaace2658e186b070" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 46777 } } peers { permanent_uuid: "d6c8a069da964e028f29b12db7ee1167" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46557 } }
I20260809 04:57:31.280998 15438 leader_election.cc:288] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [CANDIDATE]: Term 1 election: Requested vote from peers cda8a6bcca62485683954060cfbf594d (127.29.243.193:43601), d6c8a069da964e028f29b12db7ee1167 (127.29.243.194:46557)
I20260809 04:57:31.281713 14937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68de036c16304d23a7a305e2a31cfee5" candidate_uuid: "f2b42f47799d46eaaace2658e186b070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cda8a6bcca62485683954060cfbf594d"
I20260809 04:57:31.281981 15140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68de036c16304d23a7a305e2a31cfee5" candidate_uuid: "f2b42f47799d46eaaace2658e186b070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6c8a069da964e028f29b12db7ee1167"
I20260809 04:57:31.282223 14937 raft_consensus.cc:3037] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:57:31.282330 15140 raft_consensus.cc:3037] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:57:31.289041 14937 raft_consensus.cc:2454] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b42f47799d46eaaace2658e186b070 in term 1.
I20260809 04:57:31.289041 15140 raft_consensus.cc:2454] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b42f47799d46eaaace2658e186b070 in term 1.
I20260809 04:57:31.290115 15242 leader_election.cc:302] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cda8a6bcca62485683954060cfbf594d, f2b42f47799d46eaaace2658e186b070; no voters:
I20260809 04:57:31.290844 15438 raft_consensus.cc:2781] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:57:31.291684 15438 raft_consensus.cc:686] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 1 LEADER]: Becoming Leader. State: Replica: f2b42f47799d46eaaace2658e186b070, State: Running, Role: LEADER
I20260809 04:57:31.292891 15438 consensus_queue.cc:237] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda8a6bcca62485683954060cfbf594d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43601 } } peers { permanent_uuid: "f2b42f47799d46eaaace2658e186b070" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 46777 } } peers { permanent_uuid: "d6c8a069da964e028f29b12db7ee1167" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46557 } }
I20260809 04:57:31.303454 14713 catalog_manager.cc:5495] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b42f47799d46eaaace2658e186b070 (127.29.243.195). New cstate: current_term: 1 leader_uuid: "f2b42f47799d46eaaace2658e186b070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda8a6bcca62485683954060cfbf594d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b42f47799d46eaaace2658e186b070" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 46777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6c8a069da964e028f29b12db7ee1167" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46557 } health_report { overall_health: UNKNOWN } } }
I20260809 04:57:31.353201 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.343725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39618 (local address 127.29.243.195:46777)
0809 04:57:31.344081 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:31.344088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:31.344320 (+ 232us) server_negotiation.cc:407] Connection header received
0809 04:57:31.344784 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:31.344788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:31.344924 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:31.345125 (+ 201us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:57:31.345718 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.347018 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.349578 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.350330 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.350481 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.351397 (+ 916us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:57:31.352359 (+ 962us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:57:31.352545 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:31.352713 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 04:57:31.352894 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 04:57:31.368878 14937 raft_consensus.cc:1261] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d [term 1 FOLLOWER]: Refusing update from remote peer f2b42f47799d46eaaace2658e186b070: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:57:31.368878 15140 raft_consensus.cc:1261] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167 [term 1 FOLLOWER]: Refusing update from remote peer f2b42f47799d46eaaace2658e186b070: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:57:31.371253 15438 consensus_queue.cc:1035] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cda8a6bcca62485683954060cfbf594d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:57:31.371860 15441 consensus_queue.cc:1035] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c8a069da964e028f29b12db7ee1167" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:57:31.395605 15447 mvcc.cc:205] Tried to move back new op lower bound from 7316485944785596416 to 7316485944504303616. Current Snapshot: MvccSnapshot[applied={T|T < 7316485944785596416}]
I20260809 04:57:31.396049 15449 mvcc.cc:205] Tried to move back new op lower bound from 7316485944785596416 to 7316485944504303616. Current Snapshot: MvccSnapshot[applied={T|T < 7316485944785596416}]
I20260809 04:57:31.398250 15451 mvcc.cc:205] Tried to move back new op lower bound from 7316485944785596416 to 7316485944504303616. Current Snapshot: MvccSnapshot[applied={T|T < 7316485944785596416}]
I20260809 04:57:31.421599 14713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:57:31.422149 14713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:57:31.429226 14713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92d85109c4c34259bfda6b17572883c6: Sending DeleteTablet for 3 replicas of tablet 68de036c16304d23a7a305e2a31cfee5
I20260809 04:57:31.431476 14909 tablet_service.cc:1514] Processing DeleteTablet for tablet 68de036c16304d23a7a305e2a31cfee5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:57:31 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:48434
I20260809 04:57:31.431735 15115 tablet_service.cc:1514] Processing DeleteTablet for tablet 68de036c16304d23a7a305e2a31cfee5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:57:31 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:46524
I20260809 04:57:31.432046 15309 tablet_service.cc:1514] Processing DeleteTablet for tablet 68de036c16304d23a7a305e2a31cfee5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:57:31 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:39608
I20260809 04:57:31.439949 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 14811
I20260809 04:57:31.441278 15466 tablet_replica.cc:330] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: stopping tablet replica
I20260809 04:57:31.441653 15465 tablet_replica.cc:330] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: stopping tablet replica
I20260809 04:57:31.442580 15465 raft_consensus.cc:2229] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:57:31.443673 15465 raft_consensus.cc:2258] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:57:31.444576 15464 tablet_replica.cc:330] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: stopping tablet replica
I20260809 04:57:31.443770 15466 raft_consensus.cc:2229] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:57:31.445451 15464 raft_consensus.cc:2229] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:57:31.446362 15464 raft_consensus.cc:2258] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:57:31.448261 15465 ts_tablet_manager.cc:1905] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:57:31.448860 15466 raft_consensus.cc:2258] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:57:31.450807 15464 ts_tablet_manager.cc:1905] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:57:31.454711 15466 ts_tablet_manager.cc:1905] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:57:31.461457 15465 ts_tablet_manager.cc:1918] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:57:31.461759 15465 log.cc:1196] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/68de036c16304d23a7a305e2a31cfee5
I20260809 04:57:31.462795 15465 ts_tablet_manager.cc:1939] T 68de036c16304d23a7a305e2a31cfee5 P d6c8a069da964e028f29b12db7ee1167: Deleting consensus metadata
I20260809 04:57:31.463945 15464 ts_tablet_manager.cc:1918] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:57:31.464385 15464 log.cc:1196] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/68de036c16304d23a7a305e2a31cfee5
I20260809 04:57:31.465361 15464 ts_tablet_manager.cc:1939] T 68de036c16304d23a7a305e2a31cfee5 P cda8a6bcca62485683954060cfbf594d: Deleting consensus metadata
I20260809 04:57:31.466567 14694 catalog_manager.cc:4841] TS d6c8a069da964e028f29b12db7ee1167 (127.29.243.194:46557): tablet 68de036c16304d23a7a305e2a31cfee5 (table test-table [id=838e4052804b41fab7e7b700c3dec8da]) successfully deleted
I20260809 04:57:31.468807 14695 catalog_manager.cc:4841] TS cda8a6bcca62485683954060cfbf594d (127.29.243.193:43601): tablet 68de036c16304d23a7a305e2a31cfee5 (table test-table [id=838e4052804b41fab7e7b700c3dec8da]) successfully deleted
Aug 09 04:57:31 dist-test-slave-21b9 krb5kdc[14645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:57:31.481086 15466 ts_tablet_manager.cc:1918] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:57:31.481453 15466 log.cc:1196] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/68de036c16304d23a7a305e2a31cfee5
I20260809 04:57:31.482625 15466 ts_tablet_manager.cc:1939] T 68de036c16304d23a7a305e2a31cfee5 P f2b42f47799d46eaaace2658e186b070: Deleting consensus metadata
I20260809 04:57:31.486251 14693 catalog_manager.cc:4841] TS f2b42f47799d46eaaace2658e186b070 (127.29.243.195:46777): tablet 68de036c16304d23a7a305e2a31cfee5 (table test-table [id=838e4052804b41fab7e7b700c3dec8da]) successfully deleted
I20260809 04:57:31.490356 15421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.441993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48450 (local address 127.29.243.193:43601)
0809 04:57:31.442428 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:31.442434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:31.444425 (+ 1991us) server_negotiation.cc:407] Connection header received
0809 04:57:31.445426 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:31.445444 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:31.445622 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:31.445776 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:31.455456 (+ 9680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.457553 (+ 2097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.464510 (+ 6957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.465743 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.465977 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.476364 (+ 10387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:31.476397 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:31.476428 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:31.476536 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:31.483864 (+ 7328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:31.485461 (+ 1597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:31.485475 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:31.485479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:31.485803 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:31.486647 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:31.486662 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:31.486666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:31.487633 (+ 967us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:31.488266 (+ 633us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:31.488883 (+ 617us) server_negotiation.cc:299] Negotiation successful
0809 04:57:31.489323 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260809 04:57:31.492220 14999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:31.669967 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 14811
I20260809 04:57:31.731899 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 15015
Aug 09 04:57:31 dist-test-slave-21b9 krb5kdc[14645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:57:31.762054 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.733151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46540 (local address 127.29.243.194:46557)
0809 04:57:31.733474 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:31.733492 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:31.733697 (+ 205us) server_negotiation.cc:407] Connection header received
0809 04:57:31.734003 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:31.734008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:31.734132 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:31.734299 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:31.736769 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.738793 (+ 2024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.740901 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.742159 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.742406 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.749585 (+ 7179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:31.749628 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:31.749675 (+ 47us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:31.749785 (+ 110us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:31.756651 (+ 6866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:31.757853 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:31.757883 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:31.757896 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:31.758265 (+ 369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:31.758837 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:31.758844 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:31.758858 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:31.759925 (+ 1067us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:31.760519 (+ 594us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:31.760882 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 04:57:31.761316 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:57:31.763478 15201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:31.891106 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 15015
I20260809 04:57:31.952207 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 15218
Aug 09 04:57:31 dist-test-slave-21b9 krb5kdc[14645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:57:31.992731 15475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:57:31.953932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39624 (local address 127.29.243.195:46777)
0809 04:57:31.955111 (+ 1179us) server_negotiation.cc:206] Beginning negotiation
0809 04:57:31.955118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:57:31.955151 (+ 33us) server_negotiation.cc:407] Connection header received
0809 04:57:31.955246 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:57:31.955258 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:57:31.955422 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:57:31.955635 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:57:31.959818 (+ 4183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.961316 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.965523 (+ 4207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:57:31.966495 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:57:31.966705 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:57:31.980554 (+ 13849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:57:31.980587 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:57:31.980613 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:57:31.980691 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:57:31.986995 (+ 6304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:31.988392 (+ 1397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:31.988401 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:31.988403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:31.988619 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:57:31.989306 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:57:31.989312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:57:31.989314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:57:31.990016 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:57:31.990697 (+ 681us) server_negotiation.cc:1035] Waiting for connection context
0809 04:57:31.991501 (+ 804us) server_negotiation.cc:299] Negotiation successful
0809 04:57:31.992045 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":616,"threads_started":1}
I20260809 04:57:31.993821 15394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:32.176667 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 15218
I20260809 04:57:32.218571 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 14665
I20260809 04:57:32.220507 14763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:57:32.379470 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 14665
2026-08-09T04:57:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5112 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:57:32 dist-test-slave-21b9 krb5kdc[15482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:57:32 dist-test-slave-21b9 krb5kdc[15482](info): set up 2 sockets
Aug 09 04:57:32 dist-test-slave-21b9 krb5kdc[15482](info): commencing operation
krb5kdc: starting...
W20260809 04:57:34.345356 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.893s user 0.003s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:57:34 dist-test-slave-21b9 krb5kdc[15482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:57:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:57:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.243.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.243.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/rangeradmin_127.29.243.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/rangeradmin_127.29.243.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.243.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.243.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/rangerlookup_127.29.243.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/rangerlookup_127.29.243.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.243.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.243.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/HTTP_127.29.243.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/HTTP_127.29.243.212@KRBTEST.COM.keytab.
I20260809 04:57:34.766240 30671 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDjfSyW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/postgres -l logfile start
2026-08-09 04:57:37.313 UTC [15537] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 04:57:37.313 UTC [15537] LOG: listening on IPv4 address "127.29.243.212", port 44367
2026-08-09 04:57:37.316 UTC [15537] LOG: listening on Unix socket "/tmp/.s.PGSQL.44367"
2026-08-09 04:57:37.338 UTC [15540] LOG: database system was shut down at 2026-08-09 04:57:35 UTC
2026-08-09 04:57:37.341 UTC [15537] LOG: database system is ready to accept connections
I20260809 04:57:38.429361 30671 mini_postgres.cc:96] Postgres bound to 44367
2026-08-09 04:57:38.454 UTC [15555] FATAL: database "slave" does not exist
127.29.243.212:44367 - accepting connections
I20260809 04:57:38.455616 30671 mini_ranger.cc:161] Starting Ranger...
I20260809 04:57:38.487692 30671 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 04:57:38.966392 30671 mini_ranger.cc:87] Created ranger Postgres database
I20260809 04:57:38.966593 30671 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-admin
2026-08-09 04:57:39,340 [I] DB FLAVOR :POSTGRES
2026-08-09 04:57:39,340 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:57:39,340 [I] Checking connection..
2026-08-09 04:57:39,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 04:57:39,607 [I] Checking connection passed.
2026-08-09 04:57:39,607 [I] --------- Verifying version history table ---------
2026-08-09 04:57:39,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:57:39,846 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:57:39,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:57:40,095 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:57:40,095 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 04:57:40,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 04:57:40.334 UTC [15645] WARNING: there is no transaction in progress
2026-08-09 04:57:40,343 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 04:57:40,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:57:40,597 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 04:57:40,597 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:57:40,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 04:57:40,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 04:57:41,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-21b9.c.gcp-upstream.internal','N') ;"
2026-08-09 04:57:41,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:57:41,568 [I] Table x_portal_user does not exist in database ranger
2026-08-09 04:57:41,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:57:41,803 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 04:57:41,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:57:42,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:57:42,314 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 04:57:42,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 04:57:43.251 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.266 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.281 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.297 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.312 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.330 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.391 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.397 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.411 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.421 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.431 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.445 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.454 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.461 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.468 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:43.474 UTC [15803] WARNING: there is no transaction in progress
2026-08-09 04:57:44,082 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 04:57:44,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 04:57:44,337 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 04:57:44,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:57:44,581 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 04:57:44,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:57:44,833 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 04:57:44,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:57:45,080 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:57:45,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-21b9.c.gcp-upstream.internal';"
2026-08-09 04:57:45,581 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 04:57:45,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 04:57:45,842 [I] DB_PATCHES have already been applied
I20260809 04:57:45.848299 30671 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDjfSyW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 04:57:45.848420 30671 mini_ranger.cc:193] Using host: 127.29.243.212
I20260809 04:57:45.869925 30671 mini_ranger.cc:227] Ranger admin URL: http://127.29.243.212:39885
Aug 09, 2026 4:57:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:57:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:57:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:57:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:57:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:57:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:57:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 04:57:47.150290 30671 mini_ranger.cc:160] Time spent starting Ranger: real 8.695s user 0.003s sys 0.064s
Aug 09, 2026 4:58:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:58:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 4:58:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:58:07 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:58:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 04:58:08.999223 30671 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.243.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.243.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/rangerkms_127.29.243.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/rangerkms_127.29.243.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.243.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.243.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.243.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/HTTP_127.29.243.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.243.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/HTTP_127.29.243.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 04:58:09 dist-test-slave-21b9 krb5kdc[15482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251489, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 04:58:09.224004 30671 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 04:58:09.251914 30671 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 04:58:10.029975 30671 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 04:58:10.030133 30671 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-kms
I20260809 04:58:10.030284 30671 mini_ranger_kms.cc:227] Using postgres at 127.29.243.212:44367
2026-08-09 04:58:10,623 [I] DB FLAVOR :POSTGRES
2026-08-09 04:58:10,624 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:58:10,624 [I] Checking connection
2026-08-09 04:58:10,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 04:58:10,885 [I] connection success
2026-08-09 04:58:10,886 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 04:58:10,886 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 04:58:10,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 04:58:11,144 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 04:58:11,144 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:58:11,144 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 04:58:11,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44367/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 04:58:11,398 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 04:58:11.832934 30671 mini_ranger_kms.cc:310] Created kms service
I20260809 04:58:12.037792 30671 mini_ranger_kms.cc:326] Created kudu user
I20260809 04:58:12.081772 30671 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 04:58:12 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251489, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:58:12.438433 30671 mini_ranger_kms.cc:384] Added ranger policy
I20260809 04:58:12.438838 30671 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 04:58:12.438897 30671 mini_ranger_kms.cc:240] Using host: 127.29.243.212
I20260809 04:58:12.470463 30671 mini_ranger_kms.cc:276] Ranger KMS PID: 16065
I20260809 04:58:12.470640 30671 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.243.212:46961
Aug 09, 2026 4:58:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:58:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:58:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:58:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:58:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:58:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 04:58:14 dist-test-slave-21b9 krb5kdc[15482](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786251494, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.243.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:58:15.014801 30671 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.791s user 0.155s sys 0.388s
I20260809 04:58:15.015151 30671 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 04:58:15.015564 30671 mini_ranger_kms.cc:401] 127.29.243.212:46961/kms/v1/keys
Aug 09 04:58:15 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251494, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.243.212@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
Aug 09 04:58:15 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251494, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.243.212@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
Aug 09 04:58:17 dist-test-slave-21b9 krb5kdc[15482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:58:17.879520 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41735
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41735
--ranger_config_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:46961/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:58:18.669374 16135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:58:18.669822 16135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:58:18.670042 16135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:58:18.681111 16135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:58:18.681351 16135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:58:18.681538 16135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:58:18.681749 16135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:58:18.681895 16135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:58:18.697182 16135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.243.212:46961/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41735
--ranger_config_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:58:18.698832 16135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:58:18.700459 16135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:58:18.719036 16142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:58:18.719918 16143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:58:18.720183 16146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:58:18.721454 16135 server_base.cc:1034] running on GCE node
I20260809 04:58:18.722224 16135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:58:18.724033 16135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:58:18.725299 16135 hybrid_clock.cc:648] HybridClock initialized: now 1786251498725188 us; error 35 us; skew 500 ppm
I20260809 04:58:18.725878 16135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:58:18 dist-test-slave-21b9 krb5kdc[15482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251498, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:58:18.733816 16135 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:58:18.735991 16135 webserver.cc:460] Webserver started at http://127.29.243.254:37493/ using document root <none> and password file <none>
I20260809 04:58:18.736893 16135 fs_manager.cc:359] Metadata directory not provided
I20260809 04:58:18.737107 16135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:58:18.737535 16135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:58:18 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251498, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:58:18.805853 16135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "f6ccfdf7223845e193ffa209b6f9b8ea"
format_stamp: "Formatted at 2026-08-09 04:58:18 on dist-test-slave-21b9"
server_key: "91b564b5050ad30353e8daa7b109a0cd"
server_key_iv: "a4ab44bd30c3054d55482f33dc6a6317"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:18.807247 16135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "f6ccfdf7223845e193ffa209b6f9b8ea"
format_stamp: "Formatted at 2026-08-09 04:58:18 on dist-test-slave-21b9"
server_key: "91b564b5050ad30353e8daa7b109a0cd"
server_key_iv: "a4ab44bd30c3054d55482f33dc6a6317"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:18.817243 16135 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.002s
I20260809 04:58:18.843904 16158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:58:18.846594 16135 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.002s
I20260809 04:58:18.846822 16135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "f6ccfdf7223845e193ffa209b6f9b8ea"
format_stamp: "Formatted at 2026-08-09 04:58:18 on dist-test-slave-21b9"
server_key: "91b564b5050ad30353e8daa7b109a0cd"
server_key_iv: "a4ab44bd30c3054d55482f33dc6a6317"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:18.847142 16135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:58:18.888984 16135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:58:18.895468 16135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:58:18.896044 16135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:58:18.951303 16135 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:41735
I20260809 04:58:18.951529 16232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:41735 every 8 connection(s)
I20260809 04:58:18.953229 16135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:58:18.958812 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 16135
I20260809 04:58:18.959084 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:58:18.960058 16234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:58:18.960126 30671 external_mini_cluster.cc:1428] Setting key bb9f4e9f2f20f92979c2f08d9b238ae7
I20260809 04:58:18.981473 16234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea: Bootstrap starting.
I20260809 04:58:18.988617 16234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea: Neither blocks nor log segments found. Creating new log.
I20260809 04:58:18.991215 16234 log.cc:824] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea: Log is configured to *not* fsync() on all Append() calls
I20260809 04:58:18.996172 16234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea: No bootstrap required, opened a new log
Aug 09 04:58:18 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:58:19.009058 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:18.962902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34000 (local address 127.29.243.254:41735)
0809 04:58:18.980473 (+ 17571us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:18.980492 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:18.980577 (+ 85us) server_negotiation.cc:407] Connection header received
0809 04:58:18.984826 (+ 4249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:18.984864 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:18.985484 (+ 620us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:18.986047 (+ 563us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:58:18.987756 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:18.991138 (+ 3382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:18.992588 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:18.993627 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:18.993801 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:18.998609 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:58:18.998623 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:58:18.998647 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:58:18.998695 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:58:19.003656 (+ 4961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:19.004653 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:19.004663 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:19.004671 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:19.004913 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:19.005367 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:19.005371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:19.005373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:19.006133 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:58:19.006452 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:19.007043 (+ 591us) server_negotiation.cc:299] Negotiation successful
0809 04:58:19.007525 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17229,"thread_start_us":16737,"threads_started":1}
I20260809 04:58:19.011309 16234 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ccfdf7223845e193ffa209b6f9b8ea" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41735 } }
I20260809 04:58:19.012001 16234 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:58:19.012256 16234 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6ccfdf7223845e193ffa209b6f9b8ea, State: Initialized, Role: FOLLOWER
I20260809 04:58:19.013486 16234 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ccfdf7223845e193ffa209b6f9b8ea" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41735 } }
I20260809 04:58:19.013840 16234 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:58:19.014078 16234 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:58:19.014320 16234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:58:19.018373 16234 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ccfdf7223845e193ffa209b6f9b8ea" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41735 } }
I20260809 04:58:19.019030 16234 leader_election.cc:302] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6ccfdf7223845e193ffa209b6f9b8ea; no voters:
I20260809 04:58:19.020241 16234 leader_election.cc:288] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:58:19.020485 16240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:58:19.022652 16240 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [term 1 LEADER]: Becoming Leader. State: Replica: f6ccfdf7223845e193ffa209b6f9b8ea, State: Running, Role: LEADER
I20260809 04:58:19.023792 16234 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:58:19.023783 16240 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ccfdf7223845e193ffa209b6f9b8ea" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41735 } }
I20260809 04:58:19.028209 16242 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6ccfdf7223845e193ffa209b6f9b8ea. Latest consensus state: current_term: 1 leader_uuid: "f6ccfdf7223845e193ffa209b6f9b8ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ccfdf7223845e193ffa209b6f9b8ea" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41735 } } }
I20260809 04:58:19.028992 16242 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [sys.catalog]: This master's current role is: LEADER
I20260809 04:58:19.028424 16241 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6ccfdf7223845e193ffa209b6f9b8ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ccfdf7223845e193ffa209b6f9b8ea" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41735 } } }
I20260809 04:58:19.029806 16241 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea [sys.catalog]: This master's current role is: LEADER
I20260809 04:58:19.036638 16247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:58:19.046579 16247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:58:19.048194 16234 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 04:58:19.078157 16247 catalog_manager.cc:1347] Generated new cluster ID: 6f4dae41ef494a80804255c02df6d44c
I20260809 04:58:19.078320 16247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:58:19.098629 16247 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:58:19.101177 16247 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:58:19.113565 16247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea: Generated new TSK 0
I20260809 04:58:19.114851 16247 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:58:19.971851 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:46961/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41735
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:58:20 dist-test-slave-21b9 krb5kdc[15482](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786251500, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 04:58:20.603811 16294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:58:20.604216 16294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:58:20.604439 16294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:58:20.620793 16294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:58:20.621078 16294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:58:20.621411 16294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:58:20.643078 16294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.243.212:46961/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41735
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:58:20.645473 16294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:58:20.647818 16294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:58:20.675539 16308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:58:20.675822 16310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:58:20.681180 16312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:58:20.682693 16294 server_base.cc:1034] running on GCE node
I20260809 04:58:20.683565 16294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:58:20.684900 16294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:58:20.686193 16294 hybrid_clock.cc:648] HybridClock initialized: now 1786251500686013 us; error 109 us; skew 500 ppm
I20260809 04:58:20.686664 16294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:58:20 dist-test-slave-21b9 krb5kdc[15482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251500, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:58:20.691803 16294 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:58:20.694789 16294 webserver.cc:460] Webserver started at http://127.29.243.193:39211/ using document root <none> and password file <none>
I20260809 04:58:20.695786 16294 fs_manager.cc:359] Metadata directory not provided
I20260809 04:58:20.696018 16294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:58:20.696492 16294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:58:20 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251500, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:58:20.719364 16294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "514544257da8494a90c619188be5fb5d"
format_stamp: "Formatted at 2026-08-09 04:58:20 on dist-test-slave-21b9"
server_key: "29f93b936ed7c2dbcccb7daa1fc00c6e"
server_key_iv: "e60f5f9bfe5441c99c5b097835867532"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:20.720831 16294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "514544257da8494a90c619188be5fb5d"
format_stamp: "Formatted at 2026-08-09 04:58:20 on dist-test-slave-21b9"
server_key: "29f93b936ed7c2dbcccb7daa1fc00c6e"
server_key_iv: "e60f5f9bfe5441c99c5b097835867532"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:20.730897 16294 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.001s
I20260809 04:58:20.776850 16321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:58:20.783247 16294 fs_manager.cc:725] Time spent opening block manager: real 0.031s user 0.002s sys 0.015s
I20260809 04:58:20.783620 16294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "514544257da8494a90c619188be5fb5d"
format_stamp: "Formatted at 2026-08-09 04:58:20 on dist-test-slave-21b9"
server_key: "29f93b936ed7c2dbcccb7daa1fc00c6e"
server_key_iv: "e60f5f9bfe5441c99c5b097835867532"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:20.784008 16294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:58:20.836860 16294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:58:20.844403 16294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:58:20.845033 16294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:58:20.847289 16294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:58:20.850574 16294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:58:20.850729 16294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 04:58:20.850986 16294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:58:20.851130 16294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:58:20 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251500, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:58:21.074800 16294 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:45621
I20260809 04:58:21.074702 16491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:45621 every 8 connection(s)
I20260809 04:58:21.076718 16294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:58:21.079841 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 16294
I20260809 04:58:21.080080 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:58:21.080965 30671 external_mini_cluster.cc:1428] Setting key 03d311b944fde8f1e6e1578035ea2644
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Aug 09 04:58:21 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251500, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
I20260809 04:58:21.142288 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:21.098857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41735 (local address 127.29.243.193:43703)
0809 04:58:21.114014 (+ 15157us) negotiation.cc:107] Waiting for socket to connect
0809 04:58:21.114086 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 04:58:21.120931 (+ 6845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:58:21.122108 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:58:21.122120 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:58:21.122795 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:58:21.123689 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:21.123715 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:21.125206 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:21.125210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:21.126630 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:21.126651 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:21.127935 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:21.127944 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:21.128242 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:21.129056 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:58:21.129099 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:58:21.131738 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:58:21.137352 (+ 5614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:58:21.137362 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:58:21.137382 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:58:21.137939 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:58:21.138373 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:58:21.138377 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:58:21.138380 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:58:21.138617 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:58:21.139392 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:58:21.139400 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:58:21.140090 (+ 690us) client_negotiation.cc:769] Sending connection context
0809 04:58:21.140373 (+ 283us) client_negotiation.cc:240] Negotiation successful
0809 04:58:21.140925 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14692,"thread_start_us":16328,"threads_started":1}
I20260809 04:58:21.143571 16496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:21.099301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:43703 (local address 127.29.243.254:41735)
0809 04:58:21.109753 (+ 10452us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:21.109762 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:21.114256 (+ 4494us) server_negotiation.cc:407] Connection header received
0809 04:58:21.121431 (+ 7175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:21.121436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:21.121539 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:21.121639 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:58:21.123906 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:21.125042 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:21.126787 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:21.127460 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:21.127589 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:21.131974 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:58:21.131979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:58:21.131982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:58:21.132017 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:58:21.137189 (+ 5172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:21.138086 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:21.138092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:21.138096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:21.138247 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:21.138786 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:21.138791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:21.138793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:21.139137 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:58:21.139255 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:21.142996 (+ 3741us) server_negotiation.cc:299] Negotiation successful
0809 04:58:21.143211 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10288,"thread_start_us":10132,"threads_started":1}
I20260809 04:58:21.145207 16492 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41735
I20260809 04:58:21.145568 16492 heartbeater.cc:464] Registering TS with master...
I20260809 04:58:21.146449 16492 heartbeater.cc:511] Master 127.29.243.254:41735 requested a full tablet report, sending...
I20260809 04:58:21.149353 16183 ts_manager.cc:194] Registered new tserver with Master: 514544257da8494a90c619188be5fb5d (127.29.243.193:45621)
I20260809 04:58:21.151640 16183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:43703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:58:21.179870 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:46961/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41735
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:58:21 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251500, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
W20260809 04:58:21.711700 16498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:58:21.711980 16498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:58:21.712100 16498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:58:21.719504 16498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:58:21.719610 16498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:58:21.719748 16498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:58:21.732822 16498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.243.212:46961/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41735
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:58:21.734225 16498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:58:21.735718 16498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:58:21.751084 16519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:58:21.751111 16521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:58:21.755223 16523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:58:21.755719 16498 server_base.cc:1034] running on GCE node
I20260809 04:58:21.756363 16498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:58:21.757458 16498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:58:21.758693 16498 hybrid_clock.cc:648] HybridClock initialized: now 1786251501758629 us; error 56 us; skew 500 ppm
I20260809 04:58:21.759016 16498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:58:21 dist-test-slave-21b9 krb5kdc[15482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251501, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:58:21.762487 16498 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:58:21.764173 16498 webserver.cc:460] Webserver started at http://127.29.243.194:35861/ using document root <none> and password file <none>
I20260809 04:58:21.764906 16498 fs_manager.cc:359] Metadata directory not provided
I20260809 04:58:21.765069 16498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:58:21.765528 16498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:58:21 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251501, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:58:21.783632 16498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "29b9d77f05b0420381bf2cf90d47d71b"
format_stamp: "Formatted at 2026-08-09 04:58:21 on dist-test-slave-21b9"
server_key: "b117fdf919c5533772b7fd3f50b9e150"
server_key_iv: "d8fd6bc5ba2096158995deb49e5fedec"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:21.784761 16498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "29b9d77f05b0420381bf2cf90d47d71b"
format_stamp: "Formatted at 2026-08-09 04:58:21 on dist-test-slave-21b9"
server_key: "b117fdf919c5533772b7fd3f50b9e150"
server_key_iv: "d8fd6bc5ba2096158995deb49e5fedec"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:21.791188 16498 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260809 04:58:21.809836 16535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:58:21.812216 16498 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260809 04:58:21.812474 16498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "29b9d77f05b0420381bf2cf90d47d71b"
format_stamp: "Formatted at 2026-08-09 04:58:21 on dist-test-slave-21b9"
server_key: "b117fdf919c5533772b7fd3f50b9e150"
server_key_iv: "d8fd6bc5ba2096158995deb49e5fedec"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:21.812839 16498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:58:21.837638 16498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:58:21.843091 16498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:58:21.843463 16498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:58:21.844717 16498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:58:21.846525 16498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:58:21.846613 16498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:58:21.846728 16498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:58:21.846791 16498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:58:21.922394 16498 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:46857
I20260809 04:58:21.922439 16706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:46857 every 8 connection(s)
I20260809 04:58:21.924175 16498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:58:21.931099 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 16498
I20260809 04:58:21.931367 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:58:21.931885 30671 external_mini_cluster.cc:1428] Setting key 9b3dd7d333ef791d589dd7157a93cb7a
Aug 09 04:58:21 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251501, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:58:21.971644 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:21.927085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:37907 (local address 127.29.243.254:41735)
0809 04:58:21.944194 (+ 17109us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:21.944202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:21.945906 (+ 1704us) server_negotiation.cc:407] Connection header received
0809 04:58:21.951930 (+ 6024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:21.951936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:21.952026 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:21.952145 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:58:21.954146 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:21.956021 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:21.958436 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:21.959364 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:21.959495 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:21.964247 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:58:21.964252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:58:21.964256 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:58:21.964339 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:58:21.967847 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:21.968777 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:21.968782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:21.968786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:21.968952 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:21.969493 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:21.969499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:21.969502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:21.969861 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:58:21.970004 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:21.971007 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0809 04:58:21.971233 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16925,"thread_start_us":17101,"threads_started":1}
I20260809 04:58:21.974085 16717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:21.927786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41735 (local address 127.29.243.194:37907)
0809 04:58:21.945736 (+ 17950us) negotiation.cc:107] Waiting for socket to connect
0809 04:58:21.945793 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:58:21.951605 (+ 5812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:58:21.952408 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:58:21.952421 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:58:21.953122 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:58:21.953964 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:21.953991 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:21.956196 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:21.956203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:21.958270 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:21.958297 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:21.959609 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:21.959617 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:21.959922 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:21.960696 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:58:21.960729 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:58:21.964025 (+ 3296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:58:21.968036 (+ 4011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:58:21.968046 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:58:21.968066 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:58:21.968639 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:58:21.969115 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:58:21.969120 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:58:21.969124 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:58:21.969364 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:58:21.970109 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:58:21.970118 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:58:21.970552 (+ 434us) client_negotiation.cc:769] Sending connection context
0809 04:58:21.972140 (+ 1588us) client_negotiation.cc:240] Negotiation successful
0809 04:58:21.972692 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17545,"thread_start_us":19382,"threads_started":1}
I20260809 04:58:21.975993 16710 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41735
I20260809 04:58:21.976327 16710 heartbeater.cc:464] Registering TS with master...
I20260809 04:58:21.977236 16710 heartbeater.cc:511] Master 127.29.243.254:41735 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
I20260809 04:58:21.978914 16183 ts_manager.cc:194] Registered new tserver with Master: 29b9d77f05b0420381bf2cf90d47d71b (127.29.243.194:46857)
I20260809 04:58:21.980027 16183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:37907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:58:22.022867 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:46961/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41735
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:58:22.155714 16492 heartbeater.cc:503] Master 127.29.243.254:41735 was elected leader, sending a full tablet report...
W20260809 04:58:22.566536 16721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:58:22.566843 16721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:58:22.567011 16721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:58:22.576893 16721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:58:22.576999 16721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:58:22.577121 16721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:58:22.588428 16721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.243.212:46961/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41735
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:58:22.589910 16721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:58:22.591491 16721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:58:22.609100 16729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:58:22.610424 16733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:58:22.610558 16731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:58:22.611416 16721 server_base.cc:1034] running on GCE node
I20260809 04:58:22.612174 16721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:58:22.613368 16721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:58:22.614642 16721 hybrid_clock.cc:648] HybridClock initialized: now 1786251502614553 us; error 54 us; skew 500 ppm
I20260809 04:58:22.615067 16721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:58:22 dist-test-slave-21b9 krb5kdc[15482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251502, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:58:22.619580 16721 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:58:22.621462 16721 webserver.cc:460] Webserver started at http://127.29.243.195:38413/ using document root <none> and password file <none>
I20260809 04:58:22.622157 16721 fs_manager.cc:359] Metadata directory not provided
I20260809 04:58:22.622394 16721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:58:22.622874 16721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:58:22 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251502, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:58:22.638346 16721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "5b11c4795afe497aaf007cea7a1e7407"
format_stamp: "Formatted at 2026-08-09 04:58:22 on dist-test-slave-21b9"
server_key: "95e20a526103f2b77d6700dfeb0ffeeb"
server_key_iv: "d9165e149f0b9134f42c441e59b5dc21"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:22.639605 16721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "5b11c4795afe497aaf007cea7a1e7407"
format_stamp: "Formatted at 2026-08-09 04:58:22 on dist-test-slave-21b9"
server_key: "95e20a526103f2b77d6700dfeb0ffeeb"
server_key_iv: "d9165e149f0b9134f42c441e59b5dc21"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:22.646692 16721 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260809 04:58:22.664851 16741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:58:22.666602 16721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:58:22.666877 16721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "5b11c4795afe497aaf007cea7a1e7407"
format_stamp: "Formatted at 2026-08-09 04:58:22 on dist-test-slave-21b9"
server_key: "95e20a526103f2b77d6700dfeb0ffeeb"
server_key_iv: "d9165e149f0b9134f42c441e59b5dc21"
server_key_version: "kuduclusterkey@0"
I20260809 04:58:22.667191 16721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:58:22.696857 16721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:58:22.701924 16721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:58:22.702451 16721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:58:22.704093 16721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:58:22.706281 16721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:58:22.706354 16721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:58:22.706472 16721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:58:22.706523 16721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:58:22.793161 16721 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:41377
I20260809 04:58:22.793164 16912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:41377 every 8 connection(s)
I20260809 04:58:22.794934 16721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:58:22.803815 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 16721
I20260809 04:58:22.803983 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:58:22.804504 30671 external_mini_cluster.cc:1428] Setting key bfc820784b29d89d574d2af5c125d4c1
Aug 09 04:58:22 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251502, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:58:22.830559 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:22.798060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:57431 (local address 127.29.243.254:41735)
0809 04:58:22.798840 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:22.798845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:22.800559 (+ 1714us) server_negotiation.cc:407] Connection header received
0809 04:58:22.803234 (+ 2675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:22.803238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:22.803347 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:22.803529 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:58:22.805207 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.807005 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.809512 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.810368 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.810532 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:22.821087 (+ 10555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:58:22.821095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:58:22.821100 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:58:22.821140 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:58:22.826635 (+ 5495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:22.827697 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:22.827701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:22.827704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:22.827899 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:22.828428 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:22.828431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:22.828437 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:22.828870 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:58:22.829005 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:22.829895 (+ 890us) server_negotiation.cc:299] Negotiation successful
0809 04:58:22.830075 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":377,"threads_started":1}
I20260809 04:58:22.832439 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:22.798832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41735 (local address 127.29.243.195:57431)
0809 04:58:22.800369 (+ 1537us) negotiation.cc:107] Waiting for socket to connect
0809 04:58:22.800419 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:58:22.802980 (+ 2561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:58:22.803705 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:58:22.803713 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:58:22.804223 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:58:22.804979 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:22.805001 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.807151 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.807155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:22.809278 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:22.809323 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.810744 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.810753 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:22.811046 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:22.812045 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:58:22.812124 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:58:22.815375 (+ 3251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:58:22.826803 (+ 11428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:58:22.826814 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:58:22.826834 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:58:22.827571 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:58:22.828029 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:58:22.828034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:58:22.828037 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:58:22.828306 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:58:22.829024 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:58:22.829033 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:58:22.829502 (+ 469us) client_negotiation.cc:769] Sending connection context
0809 04:58:22.829827 (+ 325us) client_negotiation.cc:240] Negotiation successful
0809 04:58:22.830490 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"thread_start_us":679,"threads_started":1}
I20260809 04:58:22.834301 16913 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41735
I20260809 04:58:22.834749 16913 heartbeater.cc:464] Registering TS with master...
I20260809 04:58:22.836030 16913 heartbeater.cc:511] Master 127.29.243.254:41735 requested a full tablet report, sending...
I20260809 04:58:22.838281 16183 ts_manager.cc:194] Registered new tserver with Master: 5b11c4795afe497aaf007cea7a1e7407 (127.29.243.195:41377)
I20260809 04:58:22.839591 16183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:57431
I20260809 04:58:22.849021 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:58:22.874782 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:22.856229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34002 (local address 127.29.243.254:41735)
0809 04:58:22.856398 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:22.856403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:22.857506 (+ 1103us) server_negotiation.cc:407] Connection header received
0809 04:58:22.857709 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:22.857713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:22.857831 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:22.857927 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:58:22.860278 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.861383 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.863319 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.863975 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.864087 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:22.867517 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:58:22.867524 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:58:22.867529 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:58:22.867571 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:58:22.871438 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:22.872333 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:22.872339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:22.872342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:22.872526 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:22.872982 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:22.872987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:22.872989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:22.873362 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:58:22.873676 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:22.874100 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 04:58:22.874351 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:58:22.880367 16183 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:58:22.893146 16183 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:58:22.961756 16934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:22.914093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48062 (local address 127.29.243.195:41377)
0809 04:58:22.941256 (+ 27163us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:22.941266 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:22.941385 (+ 119us) server_negotiation.cc:407] Connection header received
0809 04:58:22.941535 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:22.941544 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:22.941828 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:22.944077 (+ 2249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:58:22.947716 (+ 3639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.949714 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.956534 (+ 6820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.959053 (+ 2519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.959591 (+ 538us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:22.959806 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:22.960801 (+ 995us) server_negotiation.cc:299] Negotiation successful
0809 04:58:22.961020 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26850,"thread_start_us":26952,"threads_started":1}
I20260809 04:58:22.962088 16933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:22.913630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41377 (local address 127.0.0.1:48062)
0809 04:58:22.915346 (+ 1716us) negotiation.cc:107] Waiting for socket to connect
0809 04:58:22.915401 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 04:58:22.915621 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:58:22.943847 (+ 28226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:58:22.943857 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:58:22.943898 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:58:22.946135 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:22.946158 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.949913 (+ 3755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.949919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:22.955213 (+ 5294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:22.955248 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.960240 (+ 4992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.960249 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:22.960545 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:22.960592 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 04:58:22.960714 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 04:58:22.960997 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1287,"thread_start_us":998,"threads_started":1}
I20260809 04:58:22.968479 16810 tablet_service.cc:1467] Processing CreateTablet for tablet 6d670bd4023d48d98acd731881309a5d (DEFAULT_TABLE table=test-table [id=d29aea208336489c98dc782ae89d30a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:58:22.970208 16935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:22.917017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45621 (local address 127.0.0.1:43326)
0809 04:58:22.951931 (+ 34914us) negotiation.cc:107] Waiting for socket to connect
0809 04:58:22.951950 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:58:22.952224 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:58:22.952945 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:58:22.952950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:58:22.952975 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:58:22.957600 (+ 4625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:22.957617 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.961261 (+ 3644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.961266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:22.966464 (+ 5198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:22.966482 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.969240 (+ 2758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.969248 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:22.969478 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:22.969518 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 04:58:22.969602 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:58:22.969744 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34593,"thread_start_us":29734,"threads_started":1}
I20260809 04:58:22.971609 16938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:22.916592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.29.243.193:45621)
0809 04:58:22.943894 (+ 27302us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:22.943902 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:22.952118 (+ 8216us) server_negotiation.cc:407] Connection header received
0809 04:58:22.952466 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:22.952474 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:22.952732 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:22.952960 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:58:22.957836 (+ 4876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.960242 (+ 2406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.966679 (+ 6437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.968491 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.968719 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:22.968903 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:22.970532 (+ 1629us) server_negotiation.cc:299] Negotiation successful
0809 04:58:22.970722 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27046,"thread_start_us":26824,"threads_started":1}
I20260809 04:58:22.972381 16810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d670bd4023d48d98acd731881309a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:58:22.977187 16396 tablet_service.cc:1467] Processing CreateTablet for tablet 6d670bd4023d48d98acd731881309a5d (DEFAULT_TABLE table=test-table [id=d29aea208336489c98dc782ae89d30a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:58:22.981081 16396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d670bd4023d48d98acd731881309a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:58:22.982534 16939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:22.916422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:46857 (local address 127.0.0.1:43416)
0809 04:58:22.947209 (+ 30787us) negotiation.cc:107] Waiting for socket to connect
0809 04:58:22.947229 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:58:22.950034 (+ 2805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:58:22.951152 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:58:22.951158 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:58:22.951187 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:58:22.969305 (+ 18118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:22.969324 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.974232 (+ 4908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.974239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:22.977587 (+ 3348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:22.977606 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.981343 (+ 3737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.981352 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:22.981654 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:22.981693 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 04:58:22.981781 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:58:22.981940 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30593,"thread_start_us":33944,"threads_started":1}
I20260809 04:58:22.983542 16710 heartbeater.cc:503] Master 127.29.243.254:41735 was elected leader, sending a full tablet report...
I20260809 04:58:22.984247 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:22.916646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.29.243.194:46857)
0809 04:58:22.944926 (+ 28280us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:22.944934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:22.949253 (+ 4319us) server_negotiation.cc:407] Connection header received
0809 04:58:22.950709 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:22.950720 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:22.950963 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:22.951194 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:58:22.969768 (+ 18574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.973958 (+ 4190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.977860 (+ 3902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:22.980321 (+ 2461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:22.980677 (+ 356us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:22.980876 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:22.982977 (+ 2101us) server_negotiation.cc:299] Negotiation successful
0809 04:58:22.983177 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28103,"thread_start_us":27850,"threads_started":1}
I20260809 04:58:22.989984 16599 tablet_service.cc:1467] Processing CreateTablet for tablet 6d670bd4023d48d98acd731881309a5d (DEFAULT_TABLE table=test-table [id=d29aea208336489c98dc782ae89d30a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:58:22.994760 16599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d670bd4023d48d98acd731881309a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:58:23.012948 16942 tablet_bootstrap.cc:492] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: Bootstrap starting.
I20260809 04:58:23.020610 16942 tablet_bootstrap.cc:654] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: Neither blocks nor log segments found. Creating new log.
I20260809 04:58:23.021756 16943 tablet_bootstrap.cc:492] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: Bootstrap starting.
I20260809 04:58:23.024678 16942 log.cc:824] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: Log is configured to *not* fsync() on all Append() calls
I20260809 04:58:23.027343 16945 tablet_bootstrap.cc:492] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: Bootstrap starting.
I20260809 04:58:23.029386 16943 tablet_bootstrap.cc:654] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: Neither blocks nor log segments found. Creating new log.
I20260809 04:58:23.032091 16943 log.cc:824] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:58:23.036751 16945 tablet_bootstrap.cc:654] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: Neither blocks nor log segments found. Creating new log.
I20260809 04:58:23.039845 16945 log.cc:824] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:58:23.040642 16942 tablet_bootstrap.cc:492] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: No bootstrap required, opened a new log
I20260809 04:58:23.041167 16942 ts_tablet_manager.cc:1397] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: Time spent bootstrapping tablet: real 0.029s user 0.021s sys 0.000s
I20260809 04:58:23.048244 16943 tablet_bootstrap.cc:492] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: No bootstrap required, opened a new log
I20260809 04:58:23.048849 16943 ts_tablet_manager.cc:1397] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: Time spent bootstrapping tablet: real 0.027s user 0.018s sys 0.008s
I20260809 04:58:23.056219 16945 tablet_bootstrap.cc:492] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: No bootstrap required, opened a new log
I20260809 04:58:23.056751 16945 ts_tablet_manager.cc:1397] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: Time spent bootstrapping tablet: real 0.030s user 0.011s sys 0.012s
I20260809 04:58:23.058866 16942 raft_consensus.cc:348] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b11c4795afe497aaf007cea7a1e7407" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41377 } } peers { permanent_uuid: "514544257da8494a90c619188be5fb5d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45621 } } peers { permanent_uuid: "29b9d77f05b0420381bf2cf90d47d71b" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46857 } }
I20260809 04:58:23.059664 16942 raft_consensus.cc:374] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:58:23.059892 16942 raft_consensus.cc:729] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b11c4795afe497aaf007cea7a1e7407, State: Initialized, Role: FOLLOWER
I20260809 04:58:23.060946 16942 consensus_queue.cc:260] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b11c4795afe497aaf007cea7a1e7407" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41377 } } peers { permanent_uuid: "514544257da8494a90c619188be5fb5d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45621 } } peers { permanent_uuid: "29b9d77f05b0420381bf2cf90d47d71b" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46857 } }
I20260809 04:58:23.067163 16943 raft_consensus.cc:348] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b11c4795afe497aaf007cea7a1e7407" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41377 } } peers { permanent_uuid: "514544257da8494a90c619188be5fb5d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45621 } } peers { permanent_uuid: "29b9d77f05b0420381bf2cf90d47d71b" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46857 } }
I20260809 04:58:23.068064 16943 raft_consensus.cc:374] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:58:23.068333 16943 raft_consensus.cc:729] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 514544257da8494a90c619188be5fb5d, State: Initialized, Role: FOLLOWER
I20260809 04:58:23.069522 16943 consensus_queue.cc:260] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b11c4795afe497aaf007cea7a1e7407" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41377 } } peers { permanent_uuid: "514544257da8494a90c619188be5fb5d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45621 } } peers { permanent_uuid: "29b9d77f05b0420381bf2cf90d47d71b" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46857 } }
I20260809 04:58:23.070705 16942 ts_tablet_manager.cc:1428] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: Time spent starting tablet: real 0.029s user 0.020s sys 0.008s
I20260809 04:58:23.071352 16913 heartbeater.cc:503] Master 127.29.243.254:41735 was elected leader, sending a full tablet report...
I20260809 04:58:23.075148 16943 ts_tablet_manager.cc:1428] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: Time spent starting tablet: real 0.026s user 0.020s sys 0.004s
I20260809 04:58:23.077631 16945 raft_consensus.cc:348] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b11c4795afe497aaf007cea7a1e7407" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41377 } } peers { permanent_uuid: "514544257da8494a90c619188be5fb5d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45621 } } peers { permanent_uuid: "29b9d77f05b0420381bf2cf90d47d71b" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46857 } }
I20260809 04:58:23.078311 16945 raft_consensus.cc:374] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:58:23.078480 16945 raft_consensus.cc:729] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29b9d77f05b0420381bf2cf90d47d71b, State: Initialized, Role: FOLLOWER
I20260809 04:58:23.079232 16945 consensus_queue.cc:260] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b11c4795afe497aaf007cea7a1e7407" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41377 } } peers { permanent_uuid: "514544257da8494a90c619188be5fb5d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45621 } } peers { permanent_uuid: "29b9d77f05b0420381bf2cf90d47d71b" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46857 } }
I20260809 04:58:23.084807 16945 ts_tablet_manager.cc:1428] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: Time spent starting tablet: real 0.028s user 0.016s sys 0.008s
W20260809 04:58:23.114701 16493 tablet.cc:2369] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:58:23.200865 16711 tablet.cc:2369] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:58:23.297767 16914 tablet.cc:2369] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:58:23.468242 16956 raft_consensus.cc:482] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:58:23.468549 16956 raft_consensus.cc:504] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b11c4795afe497aaf007cea7a1e7407" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41377 } } peers { permanent_uuid: "514544257da8494a90c619188be5fb5d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45621 } } peers { permanent_uuid: "29b9d77f05b0420381bf2cf90d47d71b" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46857 } }
I20260809 04:58:23.470346 16956 leader_election.cc:288] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 514544257da8494a90c619188be5fb5d (127.29.243.193:45621), 29b9d77f05b0420381bf2cf90d47d71b (127.29.243.194:46857)
I20260809 04:58:23.479307 16938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:23.470672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:57663 (local address 127.29.243.193:45621)
0809 04:58:23.470886 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:23.470893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:23.471223 (+ 330us) server_negotiation.cc:407] Connection header received
0809 04:58:23.471562 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:23.471568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:23.471653 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:23.471756 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:58:23.472249 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.473979 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.476924 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.478320 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.478489 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:23.478593 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:23.478810 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 04:58:23.478949 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:58:23.479326 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:23.470574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45621 (local address 127.29.243.195:57663)
0809 04:58:23.471073 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 04:58:23.471094 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:58:23.471373 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:58:23.471778 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:58:23.471781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:58:23.471802 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:58:23.472094 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:23.472106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.474127 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.474131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:23.476706 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:23.476721 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.478510 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.478515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:23.478654 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:23.478680 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:58:23.478752 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 04:58:23.478854 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":19,"thread_start_us":318,"threads_started":1}
I20260809 04:58:23.480825 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:23.471135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:47925 (local address 127.29.243.194:46857)
0809 04:58:23.471451 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:23.471457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:23.471554 (+ 97us) server_negotiation.cc:407] Connection header received
0809 04:58:23.471687 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:23.471699 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:23.471780 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:23.471869 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:58:23.472631 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.473952 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.477693 (+ 3741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.479732 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.479877 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:23.479934 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:23.480417 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 04:58:23.480552 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:58:23.480958 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:23.470606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:46857 (local address 127.29.243.195:47925)
0809 04:58:23.471472 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 04:58:23.471488 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:58:23.471582 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:58:23.471900 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:58:23.471904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:58:23.471927 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:58:23.472368 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:23.472381 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.474122 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.474128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:23.477521 (+ 3393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:58:23.477533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.479963 (+ 2430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.479971 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:58:23.480250 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:23.480276 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:58:23.480379 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 04:58:23.480472 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":41,"thread_start_us":564,"threads_started":1}
I20260809 04:58:23.481102 16422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d670bd4023d48d98acd731881309a5d" candidate_uuid: "5b11c4795afe497aaf007cea7a1e7407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "514544257da8494a90c619188be5fb5d" is_pre_election: true
I20260809 04:58:23.482200 16422 raft_consensus.cc:2454] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b11c4795afe497aaf007cea7a1e7407 in term 0.
I20260809 04:58:23.482682 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d670bd4023d48d98acd731881309a5d" candidate_uuid: "5b11c4795afe497aaf007cea7a1e7407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29b9d77f05b0420381bf2cf90d47d71b" is_pre_election: true
I20260809 04:58:23.483533 16743 leader_election.cc:302] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 514544257da8494a90c619188be5fb5d, 5b11c4795afe497aaf007cea7a1e7407; no voters:
I20260809 04:58:23.483556 16635 raft_consensus.cc:2454] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b11c4795afe497aaf007cea7a1e7407 in term 0.
I20260809 04:58:23.484079 16956 raft_consensus.cc:2781] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:58:23.484367 16956 raft_consensus.cc:482] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:58:23.484521 16956 raft_consensus.cc:3037] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:58:23.488368 16956 raft_consensus.cc:504] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b11c4795afe497aaf007cea7a1e7407" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41377 } } peers { permanent_uuid: "514544257da8494a90c619188be5fb5d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45621 } } peers { permanent_uuid: "29b9d77f05b0420381bf2cf90d47d71b" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46857 } }
I20260809 04:58:23.489233 16956 leader_election.cc:288] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [CANDIDATE]: Term 1 election: Requested vote from peers 514544257da8494a90c619188be5fb5d (127.29.243.193:45621), 29b9d77f05b0420381bf2cf90d47d71b (127.29.243.194:46857)
I20260809 04:58:23.489662 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d670bd4023d48d98acd731881309a5d" candidate_uuid: "5b11c4795afe497aaf007cea7a1e7407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29b9d77f05b0420381bf2cf90d47d71b"
I20260809 04:58:23.489702 16422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d670bd4023d48d98acd731881309a5d" candidate_uuid: "5b11c4795afe497aaf007cea7a1e7407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "514544257da8494a90c619188be5fb5d"
I20260809 04:58:23.489866 16635 raft_consensus.cc:3037] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:58:23.489998 16422 raft_consensus.cc:3037] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:58:23.493288 16635 raft_consensus.cc:2454] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b11c4795afe497aaf007cea7a1e7407 in term 1.
I20260809 04:58:23.493813 16744 leader_election.cc:302] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29b9d77f05b0420381bf2cf90d47d71b, 5b11c4795afe497aaf007cea7a1e7407; no voters:
I20260809 04:58:23.494107 16956 raft_consensus.cc:2781] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:58:23.494091 16422 raft_consensus.cc:2454] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b11c4795afe497aaf007cea7a1e7407 in term 1.
I20260809 04:58:23.494927 16956 raft_consensus.cc:686] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 1 LEADER]: Becoming Leader. State: Replica: 5b11c4795afe497aaf007cea7a1e7407, State: Running, Role: LEADER
I20260809 04:58:23.495535 16956 consensus_queue.cc:237] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b11c4795afe497aaf007cea7a1e7407" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41377 } } peers { permanent_uuid: "514544257da8494a90c619188be5fb5d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45621 } } peers { permanent_uuid: "29b9d77f05b0420381bf2cf90d47d71b" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46857 } }
I20260809 04:58:23.501607 16183 catalog_manager.cc:5495] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b11c4795afe497aaf007cea7a1e7407 (127.29.243.195). New cstate: current_term: 1 leader_uuid: "5b11c4795afe497aaf007cea7a1e7407" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b11c4795afe497aaf007cea7a1e7407" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "514544257da8494a90c619188be5fb5d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29b9d77f05b0420381bf2cf90d47d71b" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46857 } health_report { overall_health: UNKNOWN } } }
I20260809 04:58:23.539381 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:23.528459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.29.243.195:41377)
0809 04:58:23.530303 (+ 1844us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:23.530308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:23.530349 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:58:23.530436 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:23.530440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:23.530551 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:23.530721 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:58:23.531661 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.533332 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.535770 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.536612 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.536726 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:23.537501 (+ 775us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:58:23.538305 (+ 804us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:58:23.538443 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:23.538644 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 04:58:23.538725 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1712,"thread_start_us":1494,"threads_started":1}
I20260809 04:58:23.550030 16635 raft_consensus.cc:1261] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b [term 1 FOLLOWER]: Refusing update from remote peer 5b11c4795afe497aaf007cea7a1e7407: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:58:23.550057 16422 raft_consensus.cc:1261] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d [term 1 FOLLOWER]: Refusing update from remote peer 5b11c4795afe497aaf007cea7a1e7407: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:58:23.551119 16964 consensus_queue.cc:1035] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [LEADER]: Connected to new peer: Peer: permanent_uuid: "514544257da8494a90c619188be5fb5d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:58:23.551569 16956 consensus_queue.cc:1035] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29b9d77f05b0420381bf2cf90d47d71b" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:58:23.566552 16979 mvcc.cc:205] Tried to move back new op lower bound from 7316486158529150976 to 7316486158322499584. Current Snapshot: MvccSnapshot[applied={T|T < 7316486158529150976}]
I20260809 04:58:23.572407 16980 mvcc.cc:205] Tried to move back new op lower bound from 7316486158529150976 to 7316486158322499584. Current Snapshot: MvccSnapshot[applied={T|T < 7316486158529150976}]
I20260809 04:58:23.589339 16183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:58:23.589872 16183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:58:23.594108 16183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6ccfdf7223845e193ffa209b6f9b8ea: Sending DeleteTablet for 3 replicas of tablet 6d670bd4023d48d98acd731881309a5d
I20260809 04:58:23.595431 16810 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d670bd4023d48d98acd731881309a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:58:23 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:48062
I20260809 04:58:23.595497 16396 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d670bd4023d48d98acd731881309a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:58:23 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:43326
I20260809 04:58:23.595590 16599 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d670bd4023d48d98acd731881309a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:58:23 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:43416
I20260809 04:58:23.598024 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 16294
I20260809 04:58:23.611209 16993 tablet_replica.cc:330] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: stopping tablet replica
I20260809 04:58:23.612112 16993 raft_consensus.cc:2229] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:58:23.612960 16993 raft_consensus.cc:2258] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:58:23.614149 16995 tablet_replica.cc:330] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: stopping tablet replica
I20260809 04:58:23.614740 16995 raft_consensus.cc:2229] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:58:23.615253 16993 ts_tablet_manager.cc:1905] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:58:23.615633 16995 raft_consensus.cc:2258] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:58:23.615691 16996 tablet_replica.cc:330] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: stopping tablet replica
I20260809 04:58:23.616123 16996 raft_consensus.cc:2229] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:58:23.617471 16995 ts_tablet_manager.cc:1905] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:58:23.618955 16996 raft_consensus.cc:2258] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:58:23.621016 16996 ts_tablet_manager.cc:1905] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:58:23.625162 16993 ts_tablet_manager.cc:1918] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:58:23.625532 16993 log.cc:1196] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/6d670bd4023d48d98acd731881309a5d
I20260809 04:58:23.626367 16993 ts_tablet_manager.cc:1939] T 6d670bd4023d48d98acd731881309a5d P 29b9d77f05b0420381bf2cf90d47d71b: Deleting consensus metadata
Aug 09 04:58:23 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:58:23.628878 16162 catalog_manager.cc:4841] TS 29b9d77f05b0420381bf2cf90d47d71b (127.29.243.194:46857): tablet 6d670bd4023d48d98acd731881309a5d (table test-table [id=d29aea208336489c98dc782ae89d30a6]) successfully deleted
I20260809 04:58:23.633308 16995 ts_tablet_manager.cc:1918] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:58:23.633577 16995 log.cc:1196] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/6d670bd4023d48d98acd731881309a5d
I20260809 04:58:23.634284 16995 ts_tablet_manager.cc:1939] T 6d670bd4023d48d98acd731881309a5d P 5b11c4795afe497aaf007cea7a1e7407: Deleting consensus metadata
I20260809 04:58:23.634464 16996 ts_tablet_manager.cc:1918] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:58:23.634778 16996 log.cc:1196] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/6d670bd4023d48d98acd731881309a5d
I20260809 04:58:23.635628 16996 ts_tablet_manager.cc:1939] T 6d670bd4023d48d98acd731881309a5d P 514544257da8494a90c619188be5fb5d: Deleting consensus metadata
I20260809 04:58:23.636916 16161 catalog_manager.cc:4841] TS 5b11c4795afe497aaf007cea7a1e7407 (127.29.243.195:41377): tablet 6d670bd4023d48d98acd731881309a5d (table test-table [id=d29aea208336489c98dc782ae89d30a6]) successfully deleted
I20260809 04:58:23.637576 16161 catalog_manager.cc:4841] TS 514544257da8494a90c619188be5fb5d (127.29.243.193:45621): tablet 6d670bd4023d48d98acd731881309a5d (table test-table [id=d29aea208336489c98dc782ae89d30a6]) successfully deleted
I20260809 04:58:23.641074 16938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:23.599491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43328 (local address 127.29.243.193:45621)
0809 04:58:23.599708 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:23.599715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:23.608799 (+ 9084us) server_negotiation.cc:407] Connection header received
0809 04:58:23.609156 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:23.609163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:23.609262 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:23.609353 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:58:23.611536 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.612801 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.616340 (+ 3539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.618445 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.618571 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:23.630287 (+ 11716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:58:23.630304 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:58:23.630325 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:58:23.630369 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:58:23.637205 (+ 6836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:23.638270 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:23.638277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:23.638279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:23.638462 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:23.638922 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:23.638926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:23.638928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:23.639494 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:58:23.639935 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:23.640203 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 04:58:23.640423 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:58:23.642055 16483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:58:23.765513 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 16294
I20260809 04:58:23.791754 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 16498
Aug 09 04:58:23 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:58:23.811218 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:23.792620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.29.243.194:46857)
0809 04:58:23.792814 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:23.792820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:23.792886 (+ 66us) server_negotiation.cc:407] Connection header received
0809 04:58:23.793105 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:23.793109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:23.793186 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:23.793274 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:58:23.795145 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.796144 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.797451 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.797880 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.797997 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:23.805553 (+ 7556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:58:23.805568 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:58:23.805584 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:58:23.805620 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:58:23.808413 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:23.809106 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:23.809114 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:23.809116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:23.809244 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:23.809639 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:23.809643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:23.809645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:23.810019 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:58:23.810314 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:23.810618 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 04:58:23.810853 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:58:23.811937 16699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:58:23.901168 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 16498
I20260809 04:58:23.932819 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 16721
Aug 09 04:58:23 dist-test-slave-21b9 krb5kdc[15482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:58:23.951781 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:58:23.933635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.29.243.195:41377)
0809 04:58:23.933804 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:58:23.933809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:58:23.933829 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:58:23.934004 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:58:23.934008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:58:23.934074 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:58:23.934155 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:58:23.935968 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.936812 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.938643 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:58:23.939080 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:58:23.939170 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:58:23.944522 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:58:23.944535 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:58:23.944551 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:58:23.944599 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:58:23.948534 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:23.949228 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:23.949238 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:23.949242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:23.949480 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:58:23.949852 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:58:23.949857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:58:23.949861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:58:23.950336 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:58:23.950678 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 04:58:23.951060 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 04:58:23.951362 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:58:23.952683 16905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:58:24.076817 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 16721
I20260809 04:58:24.105877 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 16135
I20260809 04:58:24.107209 16227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:58:24.220674 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 16135
I20260809 04:58:24.253876 30671 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 04:58:24.589001 30671 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 04:58:24.589319 30671 mini_ranger.cc:66] Stopping Ranger...
I20260809 04:58:24.933889 30671 mini_ranger.cc:68] Stopped Ranger
2026-08-09 04:58:24.934 UTC [15537] LOG: received smart shutdown request
2026-08-09 04:58:24.938 UTC [15537] LOG: background worker "logical replication launcher" (PID 15552) exited with exit code 1
2026-08-09 04:58:24.938 UTC [15543] LOG: shutting down
2026-08-09 04:58:25.099 UTC [15537] LOG: database system is shut down
2026-08-09T04:58:25Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52802 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:58:25 dist-test-slave-21b9 krb5kdc[17022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:58:25 dist-test-slave-21b9 krb5kdc[17022](info): set up 2 sockets
Aug 09 04:58:25 dist-test-slave-21b9 krb5kdc[17022](info): commencing operation
krb5kdc: starting...
W20260809 04:58:26.464999 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:58:26 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:58:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:58:26Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.243.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.243.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/rangeradmin_127.29.243.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/rangeradmin_127.29.243.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.243.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.243.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/rangerlookup_127.29.243.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/rangerlookup_127.29.243.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.243.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.243.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/HTTP_127.29.243.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/HTTP_127.29.243.212@KRBTEST.COM.keytab.
I20260809 04:58:26.833910 30671 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDjfSyW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/postgres -l logfile start
2026-08-09 04:58:29.296 UTC [17070] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 04:58:29.296 UTC [17070] LOG: listening on IPv4 address "127.29.243.212", port 44441
2026-08-09 04:58:29.299 UTC [17070] LOG: listening on Unix socket "/tmp/.s.PGSQL.44441"
2026-08-09 04:58:29.322 UTC [17075] LOG: database system was shut down at 2026-08-09 04:58:27 UTC
2026-08-09 04:58:29.325 UTC [17070] LOG: database system is ready to accept connections
I20260809 04:58:30.414275 30671 mini_postgres.cc:96] Postgres bound to 44441
2026-08-09 04:58:30.436 UTC [17088] FATAL: database "slave" does not exist
127.29.243.212:44441 - accepting connections
I20260809 04:58:30.437415 30671 mini_ranger.cc:161] Starting Ranger...
I20260809 04:58:30.467092 30671 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 04:58:30.940778 30671 mini_ranger.cc:87] Created ranger Postgres database
I20260809 04:58:30.940934 30671 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-admin
2026-08-09 04:58:31,219 [I] DB FLAVOR :POSTGRES
2026-08-09 04:58:31,220 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:58:31,220 [I] Checking connection..
2026-08-09 04:58:31,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 04:58:31,478 [I] Checking connection passed.
2026-08-09 04:58:31,478 [I] --------- Verifying version history table ---------
2026-08-09 04:58:31,478 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:58:31,729 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:58:31,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:58:31,988 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:58:31,988 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 04:58:31,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 04:58:32.243 UTC [17187] WARNING: there is no transaction in progress
2026-08-09 04:58:32,249 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 04:58:32,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:58:32,497 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 04:58:32,498 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:58:32,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 04:58:32,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 04:58:32,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-21b9.c.gcp-upstream.internal','N') ;"
2026-08-09 04:58:33,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:58:33,464 [I] Table x_portal_user does not exist in database ranger
2026-08-09 04:58:33,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:58:33,721 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 04:58:33,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:58:33,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:58:34,210 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 04:58:34,210 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 04:58:35.129 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.144 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.159 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.174 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.189 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.208 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.259 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.266 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.279 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.288 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.298 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.311 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.319 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.325 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.332 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35.337 UTC [17342] WARNING: there is no transaction in progress
2026-08-09 04:58:35,931 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 04:58:35,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 04:58:36,170 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 04:58:36,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:58:36,427 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 04:58:36,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:58:36,687 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 04:58:36,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:58:36,938 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:58:37,185 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-21b9.c.gcp-upstream.internal';"
2026-08-09 04:58:37,422 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 04:58:37,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 04:58:37,660 [I] DB_PATCHES have already been applied
I20260809 04:58:37.666322 30671 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDjfSyW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 04:58:37.666405 30671 mini_ranger.cc:193] Using host: 127.29.243.212
I20260809 04:58:37.686655 30671 mini_ranger.cc:227] Ranger admin URL: http://127.29.243.212:51407
Aug 09, 2026 4:58:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:58:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:58:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:58:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:58:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:58:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:58:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 04:58:38.986179 30671 mini_ranger.cc:160] Time spent starting Ranger: real 8.549s user 0.003s sys 0.054s
Aug 09, 2026 4:58:57 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:58:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 4:58:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:58:57 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:58:58 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 04:58:59.382676 30671 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.243.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.243.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/rangerkms_127.29.243.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.243.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/rangerkms_127.29.243.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.243.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.243.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.243.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/HTTP_127.29.243.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.243.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/HTTP_127.29.243.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 04:58:59 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251539, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 04:58:59.624245 30671 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 04:58:59.648327 30671 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 04:59:00.445195 30671 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 04:59:00.445324 30671 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-kms
I20260809 04:59:00.445365 30671 mini_ranger_kms.cc:227] Using postgres at 127.29.243.212:44441
2026-08-09 04:59:00,801 [I] DB FLAVOR :POSTGRES
2026-08-09 04:59:00,802 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:59:00,802 [I] Checking connection
2026-08-09 04:59:00,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 04:59:01,040 [I] connection success
2026-08-09 04:59:01,040 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 04:59:01,040 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 04:59:01,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 04:59:01,272 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 04:59:01,272 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:59:01,273 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 04:59:01,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.243.212:44441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 04:59:01,552 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 04:59:01.991341 30671 mini_ranger_kms.cc:310] Created kms service
I20260809 04:59:02.208088 30671 mini_ranger_kms.cc:326] Created kudu user
I20260809 04:59:02.252902 30671 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 04:59:02 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251539, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:59:02.640261 30671 mini_ranger_kms.cc:384] Added ranger policy
I20260809 04:59:02.640607 30671 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDjfSyW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDjfSyW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 04:59:02.640679 30671 mini_ranger_kms.cc:240] Using host: 127.29.243.212
I20260809 04:59:02.666870 30671 mini_ranger_kms.cc:276] Ranger KMS PID: 17623
I20260809 04:59:02.666997 30671 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.243.212:60055
Aug 09, 2026 4:59:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:59:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:59:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:59:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:59:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:59:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 04:59:04 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786251544, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.243.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:05.231815 30671 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.608s user 0.074s sys 0.316s
I20260809 04:59:05.232095 30671 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 04:59:05.232287 30671 mini_ranger_kms.cc:401] 127.29.243.212:60055/kms/v1/keys
Aug 09 04:59:05 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251544, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.243.212@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
Aug 09 04:59:05 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251544, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.243.212@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
Aug 09 04:59:07 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:59:07.968392 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41305
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41305
--ranger_config_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:60055/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:08.334417 17710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:08.334730 17710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:08.334863 17710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:08.341930 17710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:59:08.342044 17710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:08.342118 17710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:08.342196 17710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:59:08.342264 17710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:59:08.352245 17710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.243.212:60055/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41305
--ranger_config_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:08.353714 17710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:08.355051 17710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:08.366886 17718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:08.366930 17725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:08.366882 17721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:08.368023 17710 server_base.cc:1034] running on GCE node
I20260809 04:59:08.368716 17710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:08.370077 17710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:08.371311 17710 hybrid_clock.cc:648] HybridClock initialized: now 1786251548371237 us; error 37 us; skew 500 ppm
I20260809 04:59:08.371675 17710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:08 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251548, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:08.377518 17710 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:59:08.379074 17710 webserver.cc:460] Webserver started at http://127.29.243.254:41947/ using document root <none> and password file <none>
I20260809 04:59:08.379784 17710 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:08.379899 17710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:08.380214 17710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:59:08 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251548, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:59:08.432560 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "566beff32e9e415d9263edb9ef4573d9"
format_stamp: "Formatted at 2026-08-09 04:59:08 on dist-test-slave-21b9"
server_key: "11b6b23c2e0b82583cd4ff4904c21407"
server_key_iv: "b7009d384d806892cfd6ff43d34138a5"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:08.433768 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "566beff32e9e415d9263edb9ef4573d9"
format_stamp: "Formatted at 2026-08-09 04:59:08 on dist-test-slave-21b9"
server_key: "11b6b23c2e0b82583cd4ff4904c21407"
server_key_iv: "b7009d384d806892cfd6ff43d34138a5"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:08.441098 17710 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260809 04:59:08.462178 17733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:08.464126 17710 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:59:08.464354 17710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "566beff32e9e415d9263edb9ef4573d9"
format_stamp: "Formatted at 2026-08-09 04:59:08 on dist-test-slave-21b9"
server_key: "11b6b23c2e0b82583cd4ff4904c21407"
server_key_iv: "b7009d384d806892cfd6ff43d34138a5"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:08.464644 17710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:08.501068 17710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:08.504948 17710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:08.505237 17710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:08.523130 17710 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:41305
I20260809 04:59:08.523139 17818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:41305 every 8 connection(s)
I20260809 04:59:08.524518 17710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:59:08.528476 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 17710
I20260809 04:59:08.528561 17819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:08.528689 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:59:08.529386 30671 external_mini_cluster.cc:1428] Setting key 3b9c98160421a87216fed5632ee83e2d
I20260809 04:59:08.541770 17819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9: Bootstrap starting.
Aug 09 04:59:08 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:08.545504 17819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:08.546818 17819 log.cc:824] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:08.550108 17819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9: No bootstrap required, opened a new log
I20260809 04:59:08.552863 17823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:08.531367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35892 (local address 127.29.243.254:41305)
0809 04:59:08.532157 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:08.532168 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:08.532216 (+ 48us) server_negotiation.cc:407] Connection header received
0809 04:59:08.534297 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:08.534319 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:08.534748 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:08.535083 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:08.536694 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:08.537988 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:08.539847 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:08.540489 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:08.540633 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:08.545015 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:08.545025 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:08.545045 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:08.545090 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:08.548448 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:08.549168 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:08.549177 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:08.549185 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:08.549395 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:08.549765 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:08.549769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:08.549772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:08.550465 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:08.550789 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:08.551115 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 04:59:08.551541 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":578,"thread_start_us":357,"threads_started":1}
I20260809 04:59:08.558418 17819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } }
I20260809 04:59:08.558751 17819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:08.558830 17819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 566beff32e9e415d9263edb9ef4573d9, State: Initialized, Role: FOLLOWER
I20260809 04:59:08.559412 17819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } }
I20260809 04:59:08.559645 17819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:59:08.559808 17819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:59:08.560019 17819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:08.562469 17819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } }
I20260809 04:59:08.562942 17819 leader_election.cc:302] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 566beff32e9e415d9263edb9ef4573d9; no voters:
I20260809 04:59:08.563551 17819 leader_election.cc:288] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:59:08.563676 17826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:59:08.564304 17826 raft_consensus.cc:686] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 1 LEADER]: Becoming Leader. State: Replica: 566beff32e9e415d9263edb9ef4573d9, State: Running, Role: LEADER
I20260809 04:59:08.564863 17826 consensus_queue.cc:237] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } }
I20260809 04:59:08.565511 17819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:59:08.567658 17829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 566beff32e9e415d9263edb9ef4573d9. Latest consensus state: current_term: 1 leader_uuid: "566beff32e9e415d9263edb9ef4573d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } } }
I20260809 04:59:08.567998 17828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "566beff32e9e415d9263edb9ef4573d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } } }
I20260809 04:59:08.568351 17829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:08.568364 17828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:08.569324 17842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:59:08.574934 17842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:59:08.580847 17819 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 04:59:08.598486 17842 catalog_manager.cc:1347] Generated new cluster ID: 02bb856e9c774fa5813f9c9d5427b92e
I20260809 04:59:08.598636 17842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:59:08.613413 17842 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:59:08.615501 17842 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:59:08.624753 17842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9: Generated new TSK 0
I20260809 04:59:08.625864 17842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:59:09.392135 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:60055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41305
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:59:09 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786251549, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 04:59:09.791042 17884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:09.791491 17884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:09.791651 17884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:09.800554 17884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:09.800676 17884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:09.800858 17884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:59:09.813076 17884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.243.212:60055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:09.814510 17884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:09.815912 17884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:09.831732 17899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:09.831715 17901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:09.837148 17884 server_base.cc:1034] running on GCE node
W20260809 04:59:09.837677 17905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:09.838356 17884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:09.839458 17884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:09.840693 17884 hybrid_clock.cc:648] HybridClock initialized: now 1786251549840630 us; error 49 us; skew 500 ppm
I20260809 04:59:09.841032 17884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:09 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251549, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:09.844307 17884 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:59:09.846114 17884 webserver.cc:460] Webserver started at http://127.29.243.193:39015/ using document root <none> and password file <none>
I20260809 04:59:09.846856 17884 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:09.846980 17884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:09.847414 17884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:59:09 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251549, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:59:09.867229 17884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "8e91c40086114bdfaefeef6cd27c3b23"
format_stamp: "Formatted at 2026-08-09 04:59:09 on dist-test-slave-21b9"
server_key: "6d68e7c89f79d5ac97fad2c303bc3d27"
server_key_iv: "529942398c881aa15f01e65e2aa72b9d"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:09.868376 17884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "8e91c40086114bdfaefeef6cd27c3b23"
format_stamp: "Formatted at 2026-08-09 04:59:09 on dist-test-slave-21b9"
server_key: "6d68e7c89f79d5ac97fad2c303bc3d27"
server_key_iv: "529942398c881aa15f01e65e2aa72b9d"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:09.890774 17884 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.008s sys 0.012s
I20260809 04:59:09.910821 17917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:09.912567 17884 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260809 04:59:09.912760 17884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "8e91c40086114bdfaefeef6cd27c3b23"
format_stamp: "Formatted at 2026-08-09 04:59:09 on dist-test-slave-21b9"
server_key: "6d68e7c89f79d5ac97fad2c303bc3d27"
server_key_iv: "529942398c881aa15f01e65e2aa72b9d"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:09.912979 17884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:09.954625 17884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:09.958884 17884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:09.959179 17884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:09.960256 17884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:09.965308 17884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:09.965409 17884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:09.965893 17884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:09.965963 17884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:10.035079 17884 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:46037
I20260809 04:59:10.035328 18087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:46037 every 8 connection(s)
I20260809 04:59:10.036715 17884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:59:10.042811 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 17884
I20260809 04:59:10.043038 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:59:10.044031 30671 external_mini_cluster.cc:1428] Setting key 4742cde2b553ff86bdd0f8e92996170d
Aug 09 04:59:10 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251549, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:10.072299 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:10.044982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:34049 (local address 127.29.243.254:41305)
0809 04:59:10.048196 (+ 3214us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:10.048202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:10.048221 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:59:10.048278 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:10.048284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:10.048364 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:10.048502 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:10.050396 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:10.051389 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:10.053646 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:10.054272 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:10.054402 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:10.064358 (+ 9956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:10.064367 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:10.064372 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:10.064421 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:10.068088 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:10.069164 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:10.069171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:10.069175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:10.069339 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:10.070009 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:10.070014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:10.070018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:10.070388 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:10.070516 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:10.071656 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0809 04:59:10.071911 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3083,"thread_start_us":3068,"threads_started":1}
I20260809 04:59:10.074689 18097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:10.042084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41305 (local address 127.29.243.193:34049)
0809 04:59:10.043608 (+ 1524us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:10.043673 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:10.046088 (+ 2415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:10.048932 (+ 2844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:10.048943 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:10.049609 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:10.050210 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:10.050229 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:10.051590 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:10.051595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:10.053496 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:10.053509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:10.054703 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:10.054710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:10.055002 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:10.055809 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:10.055839 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:10.063998 (+ 8159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:10.068277 (+ 4279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:10.068288 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:10.068310 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:10.068973 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:10.069544 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:10.069548 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:10.069551 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:10.069854 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:10.070633 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:10.070642 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:10.071140 (+ 498us) client_negotiation.cc:769] Sending connection context
0809 04:59:10.072609 (+ 1469us) client_negotiation.cc:240] Negotiation successful
0809 04:59:10.073219 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1179,"thread_start_us":2902,"threads_started":1}
I20260809 04:59:10.076593 18088 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41305
I20260809 04:59:10.076931 18088 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:10.077975 18088 heartbeater.cc:511] Master 127.29.243.254:41305 requested a full tablet report, sending...
I20260809 04:59:10.080714 17760 ts_manager.cc:194] Registered new tserver with Master: 8e91c40086114bdfaefeef6cd27c3b23 (127.29.243.193:46037)
I20260809 04:59:10.082743 17760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:34049
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Aug 09 04:59:10 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251549, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:59:10.137890 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:60055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41305
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:59:10 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251549, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
W20260809 04:59:10.653533 18106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:10.654065 18106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:10.654289 18106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:10.665944 18106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:10.666183 18106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:10.666435 18106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:59:10.682183 18106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.243.212:60055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:10.684319 18106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:10.686183 18106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:10.716152 18116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:10.720938 18117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:10.743631 18122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:10.745728 18106 server_base.cc:1034] running on GCE node
I20260809 04:59:10.746688 18106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:10.751693 18106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:10.753034 18106 hybrid_clock.cc:648] HybridClock initialized: now 1786251550752914 us; error 108 us; skew 500 ppm
I20260809 04:59:10.753453 18106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:10 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251550, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:10.756994 18106 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:59:10.758652 18106 webserver.cc:460] Webserver started at http://127.29.243.194:41213/ using document root <none> and password file <none>
I20260809 04:59:10.759512 18106 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:10.759636 18106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:10.759965 18106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:59:10 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251550, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:59:10.781090 18106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "1bf6136286524284bb7e896ff04bbcd3"
format_stamp: "Formatted at 2026-08-09 04:59:10 on dist-test-slave-21b9"
server_key: "227fcbaf6b5373fad1ceef2d3647538e"
server_key_iv: "eecd1e0998cbd5c3666306c7941ec984"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:10.782330 18106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "1bf6136286524284bb7e896ff04bbcd3"
format_stamp: "Formatted at 2026-08-09 04:59:10 on dist-test-slave-21b9"
server_key: "227fcbaf6b5373fad1ceef2d3647538e"
server_key_iv: "eecd1e0998cbd5c3666306c7941ec984"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:10.789785 18106 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260809 04:59:10.808421 18144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:10.810434 18106 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:59:10.810642 18106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "1bf6136286524284bb7e896ff04bbcd3"
format_stamp: "Formatted at 2026-08-09 04:59:10 on dist-test-slave-21b9"
server_key: "227fcbaf6b5373fad1ceef2d3647538e"
server_key_iv: "eecd1e0998cbd5c3666306c7941ec984"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:10.810930 18106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:10.849483 18106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:10.853470 18106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:10.853747 18106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:10.854801 18106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:10.856349 18106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:10.856422 18106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:10.856526 18106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:10.856577 18106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:10.886139 18106 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:46177
I20260809 04:59:10.886178 18320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:46177 every 8 connection(s)
I20260809 04:59:10.887450 18106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:59:10.894405 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 18106
I20260809 04:59:10.894603 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:59:10.895164 30671 external_mini_cluster.cc:1428] Setting key 0855e185417959d0fbe4c5071c6d79a4
Aug 09 04:59:10 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251550, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:10.907783 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:10.890029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:41661 (local address 127.29.243.254:41305)
0809 04:59:10.890463 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:10.890468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:10.891507 (+ 1039us) server_negotiation.cc:407] Connection header received
0809 04:59:10.893340 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:10.893345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:10.893422 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:10.893524 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:10.895387 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:10.896198 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:10.898083 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:10.898624 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:10.898744 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:10.902001 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:10.902007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:10.902012 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:10.902052 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:10.904982 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:10.905628 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:10.905633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:10.905636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:10.905777 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:10.906170 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:10.906174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:10.906176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:10.906501 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:10.906635 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:10.907230 (+ 595us) server_negotiation.cc:299] Negotiation successful
0809 04:59:10.907458 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":212,"threads_started":1}
I20260809 04:59:10.908578 18327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:10.890598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41305 (local address 127.29.243.194:41661)
0809 04:59:10.891348 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:10.891395 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:10.893056 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:10.893799 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:10.893807 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:10.894321 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:10.895129 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:10.895154 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:10.896375 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:10.896380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:10.897914 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:10.897934 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:10.898812 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:10.898817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:10.899017 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:10.899667 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:10.899690 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:10.901779 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:10.905117 (+ 3338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:10.905123 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:10.905136 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:10.905522 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:10.905872 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:10.905875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:10.905877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:10.906068 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:10.906606 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:10.906612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:10.906880 (+ 268us) client_negotiation.cc:769] Sending connection context
0809 04:59:10.907112 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 04:59:10.907509 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":270,"threads_started":1}
I20260809 04:59:10.909950 18324 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41305
I20260809 04:59:10.910243 18324 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:10.910918 18324 heartbeater.cc:511] Master 127.29.243.254:41305 requested a full tablet report, sending...
I20260809 04:59:10.912338 17760 ts_manager.cc:194] Registered new tserver with Master: 1bf6136286524284bb7e896ff04bbcd3 (127.29.243.194:46177)
I20260809 04:59:10.913111 17760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:41661
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:59:10.954279 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:60055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41305
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:59:11.086685 18088 heartbeater.cc:503] Master 127.29.243.254:41305 was elected leader, sending a full tablet report...
W20260809 04:59:11.298755 18332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:11.299086 18332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:11.299198 18332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:11.306351 18332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:11.306468 18332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:11.306594 18332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:59:11.316455 18332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.243.212:60055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:11.317804 18332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:11.319144 18332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:11.331641 18340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:11.331604 18344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:11.331676 18342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:11.332610 18332 server_base.cc:1034] running on GCE node
I20260809 04:59:11.333220 18332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:11.334158 18332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:11.335376 18332 hybrid_clock.cc:648] HybridClock initialized: now 1786251551335334 us; error 27 us; skew 500 ppm
I20260809 04:59:11.335698 18332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:11 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251551, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:11.338651 18332 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:59:11.340106 18332 webserver.cc:460] Webserver started at http://127.29.243.195:40395/ using document root <none> and password file <none>
I20260809 04:59:11.340787 18332 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:11.340910 18332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:11.341212 18332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:59:11 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251551, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:59:11.357630 18332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "1418ffc730534838a70243aa85b5107b"
format_stamp: "Formatted at 2026-08-09 04:59:11 on dist-test-slave-21b9"
server_key: "20b6922e81050a4d554899e57700f932"
server_key_iv: "dbdeae7f389e0f72bc93f19b587121b4"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:11.358440 18332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "1418ffc730534838a70243aa85b5107b"
format_stamp: "Formatted at 2026-08-09 04:59:11 on dist-test-slave-21b9"
server_key: "20b6922e81050a4d554899e57700f932"
server_key_iv: "dbdeae7f389e0f72bc93f19b587121b4"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:11.364079 18332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 04:59:11.380427 18355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:11.381982 18332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:59:11.382150 18332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "1418ffc730534838a70243aa85b5107b"
format_stamp: "Formatted at 2026-08-09 04:59:11 on dist-test-slave-21b9"
server_key: "20b6922e81050a4d554899e57700f932"
server_key_iv: "dbdeae7f389e0f72bc93f19b587121b4"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:11.382367 18332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:11.412242 18332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:11.416013 18332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:11.416291 18332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:11.417328 18332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:11.418732 18332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:11.418804 18332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:11.418919 18332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:11.418972 18332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:11.448427 18332 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:44543
I20260809 04:59:11.448457 18541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:44543 every 8 connection(s)
I20260809 04:59:11.449774 18332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:59:11.450790 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 18332
I20260809 04:59:11.450981 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:59:11.451496 30671 external_mini_cluster.cc:1428] Setting key 0a9cb804ab2f20677f62b3cf5d2ad318
Aug 09 04:59:11 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251551, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:11.468892 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:11.452493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:57417 (local address 127.29.243.254:41305)
0809 04:59:11.452988 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:11.452994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:11.454016 (+ 1022us) server_negotiation.cc:407] Connection header received
0809 04:59:11.455869 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:11.455874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:11.455967 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:11.456061 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:11.457444 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:11.458164 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:11.459718 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:11.460151 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:11.460258 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:11.463348 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:11.463353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:11.463356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:11.463388 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:11.465917 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:11.466658 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:11.466664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:11.466668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:11.466810 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:11.467200 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:11.467205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:11.467208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:11.467537 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:11.467669 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:11.468307 (+ 638us) server_negotiation.cc:299] Negotiation successful
0809 04:59:11.468534 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":238,"threads_started":1}
I20260809 04:59:11.469827 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:11.453092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41305 (local address 127.29.243.195:57417)
0809 04:59:11.453875 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:11.453915 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:11.455619 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:11.456306 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:11.456315 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:11.456727 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:11.457287 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:11.457303 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:11.458307 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:11.458311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:11.459581 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:11.459601 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:11.460360 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:11.460367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:11.460590 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:11.461136 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:11.461158 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:11.463117 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:11.466051 (+ 2934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:11.466059 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:11.466075 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:11.466557 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:11.466907 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:11.466920 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:11.466924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:11.467096 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:11.467658 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:11.467666 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:11.467979 (+ 313us) client_negotiation.cc:769] Sending connection context
0809 04:59:11.468212 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 04:59:11.468708 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":298,"threads_started":1}
I20260809 04:59:11.471170 18543 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41305
I20260809 04:59:11.471478 18543 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:11.472172 18543 heartbeater.cc:511] Master 127.29.243.254:41305 requested a full tablet report, sending...
I20260809 04:59:11.473585 17760 ts_manager.cc:194] Registered new tserver with Master: 1418ffc730534838a70243aa85b5107b (127.29.243.195:44543)
I20260809 04:59:11.474377 17760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:57417
I20260809 04:59:11.478958 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:59:11.480665 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 17884
Aug 09 04:59:11 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:59:11.497383 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:11.481453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52902 (local address 127.29.243.193:46037)
0809 04:59:11.482031 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:11.482037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:11.482212 (+ 175us) server_negotiation.cc:407] Connection header received
0809 04:59:11.482382 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:11.482389 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:11.482594 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:11.482891 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:11.484304 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:11.485313 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:11.486839 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:11.487441 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:11.487550 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:11.491752 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:11.491764 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:11.491778 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:11.491812 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:11.494334 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:11.495070 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:11.495078 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:11.495080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:11.495232 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:11.495607 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:11.495612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:11.495615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:11.495950 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:11.496256 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:11.496611 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 04:59:11.496874 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":273,"threads_started":1}
I20260809 04:59:11.498694 18083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:11.581300 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 17884
I20260809 04:59:11.606220 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 18106
Aug 09 04:59:11 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:59:11.623659 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:11.607315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55972 (local address 127.29.243.194:46177)
0809 04:59:11.607941 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:11.607949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:11.607978 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:59:11.608052 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:11.608059 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:11.608287 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:11.608428 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:11.609616 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:11.610661 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:11.612154 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:11.612735 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:11.612863 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:11.617496 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:11.617509 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:11.617525 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:11.617568 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:11.620504 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:11.621315 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:11.621325 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:11.621328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:11.621476 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:11.621879 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:11.621884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:11.621886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:11.622279 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:11.622598 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:11.622975 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 04:59:11.623253 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":321,"threads_started":1}
I20260809 04:59:11.625051 18314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:11.706480 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 18106
I20260809 04:59:11.732296 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 18332
Aug 09 04:59:11 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:59:11.748799 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:11.733445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54256 (local address 127.29.243.195:44543)
0809 04:59:11.734126 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:11.734133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:11.734164 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:59:11.734228 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:11.734235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:11.734460 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:11.734610 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:11.736031 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:11.737067 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:11.738439 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:11.738997 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:11.739114 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:11.743704 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:11.743716 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:11.743734 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:11.743775 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:11.746377 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:11.747031 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:11.747038 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:11.747040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:11.747164 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:11.747514 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:11.747517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:11.747519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:11.747845 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:11.748093 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:11.748296 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 04:59:11.748504 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":354,"threads_started":1}
I20260809 04:59:11.750142 18534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:11.832180 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 18332
I20260809 04:59:11.855911 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 17710
I20260809 04:59:11.857270 17812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:11.963014 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 17710
I20260809 04:59:11.996004 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41305
--webserver_interface=127.29.243.254
--webserver_port=41947
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41305
--ranger_config_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:60055/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.243.212:60055/kms with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:12.336108 18562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:12.336396 18562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:12.336493 18562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:12.343502 18562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:59:12.343608 18562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:12.343653 18562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:12.343710 18562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:59:12.343756 18562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:59:12.353713 18562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.243.212:60055/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41305
--ranger_config_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=41947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:12.355001 18562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:12.356343 18562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:12.368577 18567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:12.368544 18569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:12.368498 18572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:12.369395 18562 server_base.cc:1034] running on GCE node
I20260809 04:59:12.370157 18562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:12.371757 18562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:12.372969 18562 hybrid_clock.cc:648] HybridClock initialized: now 1786251552372926 us; error 42 us; skew 500 ppm
I20260809 04:59:12.373308 18562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:12 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251552, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:12.376308 18562 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:59:12.377712 18562 webserver.cc:460] Webserver started at http://127.29.243.254:41947/ using document root <none> and password file <none>
I20260809 04:59:12.378355 18562 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:12.378479 18562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:59:12 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251552, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:59:12.419381 18562 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 04:59:12.422755 18586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:12.424515 18562 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260809 04:59:12.424733 18562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "566beff32e9e415d9263edb9ef4573d9"
format_stamp: "Formatted at 2026-08-09 04:59:08 on dist-test-slave-21b9"
server_key: "11b6b23c2e0b82583cd4ff4904c21407"
server_key_iv: "b7009d384d806892cfd6ff43d34138a5"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:12.425740 18562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:12.455888 18562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:12.460005 18562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:12.460279 18562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:12.477759 18562 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:41305
I20260809 04:59:12.477783 18678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:41305 every 8 connection(s)
I20260809 04:59:12.479296 18562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:59:12.486076 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 18562
I20260809 04:59:12.486619 18679 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:59:12.487165 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:60055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:46037
--local_ip_for_outbound_sockets=127.29.243.193
--tserver_master_addrs=127.29.243.254:41305
--webserver_port=39015
--webserver_interface=127.29.243.193
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.243.212:60055/kms with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:59:12.491130 18679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9: Bootstrap starting.
I20260809 04:59:12.511812 18679 log.cc:824] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:12.521039 18679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:59:12.521659 18679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9: Bootstrap complete.
I20260809 04:59:12.530993 18679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } }
I20260809 04:59:12.531965 18679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 566beff32e9e415d9263edb9ef4573d9, State: Initialized, Role: FOLLOWER
I20260809 04:59:12.532598 18679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } }
I20260809 04:59:12.532824 18679 raft_consensus.cc:388] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:59:12.532946 18679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:59:12.533114 18679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:59:12.537019 18679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } }
I20260809 04:59:12.537495 18679 leader_election.cc:302] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 566beff32e9e415d9263edb9ef4573d9; no voters:
I20260809 04:59:12.538080 18679 leader_election.cc:288] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:59:12.538296 18685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:59:12.539332 18685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [term 2 LEADER]: Becoming Leader. State: Replica: 566beff32e9e415d9263edb9ef4573d9, State: Running, Role: LEADER
I20260809 04:59:12.539968 18685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } }
I20260809 04:59:12.540342 18679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:59:12.542757 18690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "566beff32e9e415d9263edb9ef4573d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } } }
I20260809 04:59:12.542953 18691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 566beff32e9e415d9263edb9ef4573d9. Latest consensus state: current_term: 2 leader_uuid: "566beff32e9e415d9263edb9ef4573d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566beff32e9e415d9263edb9ef4573d9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41305 } } }
I20260809 04:59:12.543170 18690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:12.543200 18691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:12.555151 18694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:59:12.565690 18694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:59:12.567292 18694 catalog_manager.cc:1259] Loaded cluster ID: 02bb856e9c774fa5813f9c9d5427b92e
I20260809 04:59:12.567480 18694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:59:12.571738 18694 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:59:12.573374 18694 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9: Loaded TSK: 0
I20260809 04:59:12.575034 18694 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:59:12.577309 18679 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 04:59:12.927659 18682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:12.927949 18682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:12.928108 18682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:12.937461 18682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:12.937670 18682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:12.937893 18682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:59:12.952956 18682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.243.212:60055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:46037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=39015
--tserver_master_addrs=127.29.243.254:41305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:12.954952 18682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:12.956861 18682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:12.987923 18739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:12.992033 18738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:12.993312 18741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:12.994016 18682 server_base.cc:1034] running on GCE node
I20260809 04:59:12.994928 18682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:12.997370 18682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:12.998643 18682 hybrid_clock.cc:648] HybridClock initialized: now 1786251552998568 us; error 65 us; skew 500 ppm
I20260809 04:59:12.998976 18682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:12 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251552, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:13.003288 18682 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:59:13.009473 18682 webserver.cc:460] Webserver started at http://127.29.243.193:39015/ using document root <none> and password file <none>
I20260809 04:59:13.010281 18682 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:13.010439 18682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:59:13 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251552, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:59:13.049724 18682 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.003s sys 0.003s
I20260809 04:59:13.076086 18749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:13.078719 18682 fs_manager.cc:725] Time spent opening block manager: real 0.026s user 0.002s sys 0.017s
I20260809 04:59:13.079057 18682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "8e91c40086114bdfaefeef6cd27c3b23"
format_stamp: "Formatted at 2026-08-09 04:59:09 on dist-test-slave-21b9"
server_key: "6d68e7c89f79d5ac97fad2c303bc3d27"
server_key_iv: "529942398c881aa15f01e65e2aa72b9d"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:13.080649 18682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:13.102191 18682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:13.107167 18682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:13.107476 18682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:13.108546 18682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:13.110109 18682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:13.110203 18682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:13.110314 18682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:13.110396 18682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:13.177659 18682 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:46037
I20260809 04:59:13.177731 18941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:46037 every 8 connection(s)
I20260809 04:59:13.179625 18682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:59:13.185905 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 18682
I20260809 04:59:13.188858 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:60055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:46177
--local_ip_for_outbound_sockets=127.29.243.194
--tserver_master_addrs=127.29.243.254:41305
--webserver_port=41213
--webserver_interface=127.29.243.194
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.243.212:60055/kms with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:59:13 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251552, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:13.217574 18945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:13.183069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41305 (local address 127.29.243.193:56921)
0809 04:59:13.183932 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:13.183979 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:13.186144 (+ 2165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:13.188672 (+ 2528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:13.188683 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:13.189641 (+ 958us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:13.190502 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:13.190535 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:13.193506 (+ 2971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:13.193513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:13.195657 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:13.195677 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:13.197232 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:13.197240 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:13.197558 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:13.198426 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:13.198452 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:13.200674 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:13.208295 (+ 7621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:13.208308 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:13.208329 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:13.209153 (+ 824us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:13.210300 (+ 1147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:13.210306 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:13.210310 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:13.210583 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:13.214550 (+ 3967us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:13.214560 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:13.215056 (+ 496us) client_negotiation.cc:769] Sending connection context
0809 04:59:13.215409 (+ 353us) client_negotiation.cc:240] Negotiation successful
0809 04:59:13.216054 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":351,"threads_started":1}
I20260809 04:59:13.221812 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:13.183095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:56921 (local address 127.29.243.254:41305)
0809 04:59:13.186971 (+ 3876us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:13.186990 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:13.187041 (+ 51us) server_negotiation.cc:407] Connection header received
0809 04:59:13.187527 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:13.187545 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:13.188032 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:13.190802 (+ 2770us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:13.191307 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:13.193277 (+ 1970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:13.195897 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:13.196951 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:13.200884 (+ 3933us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:13.201264 (+ 380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:13.201275 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:13.201298 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:13.201344 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:13.207856 (+ 6512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:13.209368 (+ 1512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:13.209379 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:13.209388 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:13.210128 (+ 740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:13.210735 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:13.210740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:13.210744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:13.214052 (+ 3308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:13.219514 (+ 5462us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:13.219854 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 04:59:13.220372 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3593,"thread_start_us":3358,"threads_started":1}
I20260809 04:59:13.226061 18943 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41305
I20260809 04:59:13.226434 18943 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:13.227474 18943 heartbeater.cc:511] Master 127.29.243.254:41305 requested a full tablet report, sending...
W20260809 04:59:13.231401 18943 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:41305 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:59:13.231652 18943 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:13.231979 18943 heartbeater.cc:511] Master 127.29.243.254:41305 requested a full tablet report, sending...
I20260809 04:59:13.233331 18943 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:13.233632 18943 heartbeater.cc:511] Master 127.29.243.254:41305 requested a full tablet report, sending...
W20260809 04:59:13.234829 18943 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:59:13.609984 18948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:13.610296 18948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:13.610433 18948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:13.618731 18948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:13.618865 18948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:13.619046 18948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:59:13.629122 18948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.243.212:60055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:46177
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=41213
--tserver_master_addrs=127.29.243.254:41305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:13.630582 18948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:13.632031 18948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:13.659857 18972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:13.668850 18973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:13.683434 18975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:13.683985 18948 server_base.cc:1034] running on GCE node
I20260809 04:59:13.684734 18948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:13.686117 18948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:13.687354 18948 hybrid_clock.cc:648] HybridClock initialized: now 1786251553687319 us; error 68 us; skew 500 ppm
I20260809 04:59:13.687701 18948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:13 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251553, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:13.694777 18948 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:59:13.719559 18948 webserver.cc:460] Webserver started at http://127.29.243.194:41213/ using document root <none> and password file <none>
I20260809 04:59:13.720613 18948 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:13.720783 18948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:59:13 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251553, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:59:13.750682 18948 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 04:59:13.754678 18999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:13.756541 18948 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260809 04:59:13.756801 18948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "1bf6136286524284bb7e896ff04bbcd3"
format_stamp: "Formatted at 2026-08-09 04:59:10 on dist-test-slave-21b9"
server_key: "227fcbaf6b5373fad1ceef2d3647538e"
server_key_iv: "eecd1e0998cbd5c3666306c7941ec984"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:13.758286 18948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 09 04:59:13 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786251553, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:13.781709 18948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:13.786974 18948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:13.787297 18948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:13.788558 18948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:13.790369 18948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:13.790548 18948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:13.790704 18948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:13.790820 18948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:13.846195 18948 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:46177
I20260809 04:59:13.846249 19176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:46177 every 8 connection(s)
I20260809 04:59:13.847934 18948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:59:13.849696 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 18948
I20260809 04:59:13.850519 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.243.212:60055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:44543
--local_ip_for_outbound_sockets=127.29.243.195
--tserver_master_addrs=127.29.243.254:41305
--webserver_port=40395
--webserver_interface=127.29.243.195
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.243.212:60055/kms with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:59:13 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251553, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:13.893966 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:13.856351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41305 (local address 127.29.243.194:38969)
0809 04:59:13.871755 (+ 15404us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:13.871809 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:13.874608 (+ 2799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:13.875473 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:13.875482 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:13.875934 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:13.876531 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:13.876548 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:13.877630 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:13.877635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:13.879625 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:13.879641 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:13.880929 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:13.880938 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:13.881235 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:13.882081 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:13.882122 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:13.884878 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:13.888997 (+ 4119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:13.889007 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:13.889026 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:13.889641 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:13.890086 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:13.890092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:13.890095 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:13.890352 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:13.891023 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:13.891031 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:13.891547 (+ 516us) client_negotiation.cc:769] Sending connection context
0809 04:59:13.891873 (+ 326us) client_negotiation.cc:240] Negotiation successful
0809 04:59:13.892470 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15078,"thread_start_us":14835,"threads_started":1}
I20260809 04:59:13.895476 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:13.855581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:38969 (local address 127.29.243.254:41305)
0809 04:59:13.871031 (+ 15450us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:13.871039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:13.872310 (+ 1271us) server_negotiation.cc:407] Connection header received
0809 04:59:13.874956 (+ 2646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:13.874962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:13.875052 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:13.875149 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:13.876708 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:13.877472 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:13.879805 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:13.880465 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:13.880564 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:13.885174 (+ 4610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:13.885182 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:13.885187 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:13.885229 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:13.888831 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:13.889800 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:13.889806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:13.889810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:13.889965 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:13.890497 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:13.890503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:13.890506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:13.890882 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:13.891032 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:13.894842 (+ 3810us) server_negotiation.cc:299] Negotiation successful
0809 04:59:13.895092 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15271,"thread_start_us":15469,"threads_started":1}
I20260809 04:59:13.897049 19177 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41305
I20260809 04:59:13.897368 19177 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:13.898260 19177 heartbeater.cc:511] Master 127.29.243.254:41305 requested a full tablet report, sending...
I20260809 04:59:13.900650 18611 ts_manager.cc:194] Registered new tserver with Master: 1bf6136286524284bb7e896ff04bbcd3 (127.29.243.194:46177)
I20260809 04:59:13.902861 18611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:38969
Aug 09 04:59:14 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251553, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:59:14.234997 18943 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:14.235496 18943 heartbeater.cc:511] Master 127.29.243.254:41305 requested a full tablet report, sending...
I20260809 04:59:14.236810 18611 ts_manager.cc:194] Registered new tserver with Master: 8e91c40086114bdfaefeef6cd27c3b23 (127.29.243.193:46037)
I20260809 04:59:14.238085 18611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:56921
W20260809 04:59:14.320396 19182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:14.320722 19182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:14.320910 19182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:14.328372 19182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:14.328528 19182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:14.328687 19182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:59:14.339241 19182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.243.212:60055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:44543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=40395
--tserver_master_addrs=127.29.243.254:41305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:14.340786 19182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:14.342221 19182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:14.356374 19191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:14.356405 19194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:14.356685 19190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:14.358053 19182 server_base.cc:1034] running on GCE node
I20260809 04:59:14.358731 19182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:14.360014 19182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:14.361331 19182 hybrid_clock.cc:648] HybridClock initialized: now 1786251554361223 us; error 92 us; skew 500 ppm
I20260809 04:59:14.361733 19182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:14 dist-test-slave-21b9 krb5kdc[17022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251554, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:14.365986 19182 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:59:14.371657 19182 webserver.cc:460] Webserver started at http://127.29.243.195:40395/ using document root <none> and password file <none>
I20260809 04:59:14.372692 19182 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:14.372928 19182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:59:14 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251554, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:59:14.411999 19182 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.012s sys 0.000s
I20260809 04:59:14.415936 19210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:14.417811 19182 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:59:14.417997 19182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "1418ffc730534838a70243aa85b5107b"
format_stamp: "Formatted at 2026-08-09 04:59:11 on dist-test-slave-21b9"
server_key: "20b6922e81050a4d554899e57700f932"
server_key_iv: "dbdeae7f389e0f72bc93f19b587121b4"
server_key_version: "kuduclusterkey@0"
I20260809 04:59:14.419499 19182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:14.444166 19182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:14.449327 19182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:14.449676 19182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:14.451783 19182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:14.454782 19182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:14.454901 19182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:14.455065 19182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:14.455147 19182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:59:14 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251553, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for HTTP/127.29.243.212@KRBTEST.COM
I20260809 04:59:14.604502 19378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:44543 every 8 connection(s)
I20260809 04:59:14.604895 19182 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:44543
I20260809 04:59:14.607118 19182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:59:14.613765 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 19182
Aug 09 04:59:14 dist-test-slave-21b9 krb5kdc[17022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251554, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:14.680862 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:14.635869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41305 (local address 127.29.243.195:52623)
0809 04:59:14.647118 (+ 11249us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:14.647173 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:14.649747 (+ 2574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:14.656161 (+ 6414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:14.656175 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:14.657094 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:14.658049 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:14.658082 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.659658 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.659663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:14.661792 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:14.661818 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.663044 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.663051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:14.663375 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:14.664269 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:14.664301 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:14.667297 (+ 2996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:14.675801 (+ 8504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:14.675812 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:14.675836 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:14.676498 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:14.676949 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:14.676954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:14.676958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:14.677224 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:14.677894 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:14.677903 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:14.678403 (+ 500us) client_negotiation.cc:769] Sending connection context
0809 04:59:14.678732 (+ 329us) client_negotiation.cc:240] Negotiation successful
0809 04:59:14.679395 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10915,"thread_start_us":10631,"threads_started":1}
I20260809 04:59:14.682804 19381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:14.636510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:52623 (local address 127.29.243.254:41305)
0809 04:59:14.650943 (+ 14433us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:14.650950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:14.650970 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:59:14.651056 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:14.651063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:14.651160 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:14.651300 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:14.658285 (+ 6985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.659467 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.661976 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.662600 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.662713 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:14.667590 (+ 4877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:14.667596 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:14.667601 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:14.667641 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:14.675614 (+ 7973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:14.676670 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:14.676674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:14.676678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:14.676816 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:14.677386 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:14.677391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:14.677394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:14.677742 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:14.682000 (+ 4258us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:14.682150 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 04:59:14.682404 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14319,"thread_start_us":13774,"threads_started":1}
I20260809 04:59:14.684916 19379 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41305
I20260809 04:59:14.685285 19379 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:14.686190 19379 heartbeater.cc:511] Master 127.29.243.254:41305 requested a full tablet report, sending...
I20260809 04:59:14.688063 18611 ts_manager.cc:194] Registered new tserver with Master: 1418ffc730534838a70243aa85b5107b (127.29.243.195:44543)
I20260809 04:59:14.689234 18611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:52623
I20260809 04:59:14.694410 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:14.615579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.29.243.254:41305)
0809 04:59:14.650545 (+ 34966us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:14.650553 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:14.656075 (+ 5522us) server_negotiation.cc:407] Connection header received
0809 04:59:14.656150 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:14.656155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:14.656230 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:14.656325 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:14.658699 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.659827 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.663395 (+ 3568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.664002 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.664103 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:14.669596 (+ 5493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:14.669603 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:14.669608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:14.669660 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:14.673116 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:14.674348 (+ 1232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:14.674354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:14.674370 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:14.674553 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:14.692442 (+ 17889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:14.692448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:14.692452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:14.692812 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:14.693082 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:14.693761 (+ 679us) server_negotiation.cc:299] Negotiation successful
0809 04:59:14.694024 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34832,"thread_start_us":35103,"threads_started":1}
I20260809 04:59:14.698666 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:59:14.795163 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:14.757104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35910 (local address 127.29.243.254:41305)
0809 04:59:14.757330 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:14.757336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:14.777320 (+ 19984us) server_negotiation.cc:407] Connection header received
0809 04:59:14.777708 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:14.777715 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:14.777836 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:14.777939 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:14.780552 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.781691 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.783751 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.784447 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.784556 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:14.788307 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:14.788314 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:14.788319 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:14.788364 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:14.791890 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:14.792738 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:14.792744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:14.792748 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:14.792919 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:14.793400 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:14.793406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:14.793409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:14.793749 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:14.794000 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:14.794569 (+ 569us) server_negotiation.cc:299] Negotiation successful
0809 04:59:14.794810 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:59:14.802481 18611 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:59:14.814363 18611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:59:14.884366 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:14.850381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.29.243.194:46177)
0809 04:59:14.870515 (+ 20134us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:14.870523 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:14.875434 (+ 4911us) server_negotiation.cc:407] Connection header received
0809 04:59:14.875565 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:14.875572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:14.875801 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:14.876008 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:14.877062 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.879129 (+ 2067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.881584 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.883151 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.883345 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:14.883547 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:14.883881 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 04:59:14.884040 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19981,"thread_start_us":19854,"threads_started":1}
I20260809 04:59:14.884490 19418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:14.850056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:46177 (local address 127.0.0.1:55984)
0809 04:59:14.874897 (+ 24841us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:14.874914 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:14.875035 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:14.876054 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:14.876060 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:14.876092 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:14.876800 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:14.876815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.879321 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.879326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:14.881426 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:14.881439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.883386 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.883392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:14.883583 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:14.883629 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 04:59:14.883758 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 04:59:14.883963 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24713,"mutex_wait_us":27,"spinlock_wait_cycles":9984,"thread_start_us":11982,"threads_started":1}
I20260809 04:59:14.888131 19421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:14.849604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:46037 (local address 127.0.0.1:52910)
0809 04:59:14.874206 (+ 24602us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:14.874225 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:14.874392 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:14.876276 (+ 1884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:14.876281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:14.876319 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:14.876948 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:14.876964 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.878891 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.878896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:14.882675 (+ 3779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:14.882690 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.886041 (+ 3351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.887355 (+ 1314us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:14.887507 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:14.887532 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:59:14.887612 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:59:14.887750 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24446,"mutex_wait_us":6,"thread_start_us":24284,"threads_started":1}
I20260809 04:59:14.888615 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:14.849802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52910 (local address 127.29.243.193:46037)
0809 04:59:14.870084 (+ 20282us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:14.870095 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:14.874344 (+ 4249us) server_negotiation.cc:407] Connection header received
0809 04:59:14.874570 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:14.874580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:14.874879 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:14.875104 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:14.877155 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.878723 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.882867 (+ 4144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.884274 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.884475 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:14.884631 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:14.887714 (+ 3083us) server_negotiation.cc:299] Negotiation successful
0809 04:59:14.887903 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20059,"thread_start_us":20714,"threads_started":1}
I20260809 04:59:14.889662 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:14.848114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:44543 (local address 127.0.0.1:54268)
0809 04:59:14.862578 (+ 14464us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:14.862628 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:14.862845 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:14.876220 (+ 13375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:14.876224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:14.876260 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:14.876803 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:14.876833 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.879919 (+ 3086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.879924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:14.882028 (+ 2104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:14.882045 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.889042 (+ 6997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.889049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:14.889209 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:14.889233 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:59:14.889285 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 04:59:14.889384 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14102,"mutex_wait_us":18,"spinlock_wait_cycles":7296,"thread_start_us":26496,"threads_started":1}
I20260809 04:59:14.889483 19077 tablet_service.cc:1467] Processing CreateTablet for tablet 061d9d7153e8479195e5828cb5d5510e (DEFAULT_TABLE table=test-table [id=2026f7bf8a5e4211b167b7ca35eaa532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:14.890707 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:14.848417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54268 (local address 127.29.243.195:44543)
0809 04:59:14.875615 (+ 27198us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:14.875623 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:14.875654 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:59:14.875774 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:14.875781 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:14.876009 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:14.876201 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:14.877740 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.879302 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.884778 (+ 5476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:14.888784 (+ 4006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:14.889955 (+ 1171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:14.890125 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:14.890207 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 04:59:14.890381 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27021,"thread_start_us":26862,"threads_started":1}
I20260809 04:59:14.892356 19077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061d9d7153e8479195e5828cb5d5510e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:14.893694 18819 tablet_service.cc:1467] Processing CreateTablet for tablet 061d9d7153e8479195e5828cb5d5510e (DEFAULT_TABLE table=test-table [id=2026f7bf8a5e4211b167b7ca35eaa532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:14.897389 19274 tablet_service.cc:1467] Processing CreateTablet for tablet 061d9d7153e8479195e5828cb5d5510e (DEFAULT_TABLE table=test-table [id=2026f7bf8a5e4211b167b7ca35eaa532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:14.898437 18819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061d9d7153e8479195e5828cb5d5510e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:14.898722 19274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061d9d7153e8479195e5828cb5d5510e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:14.909128 19177 heartbeater.cc:503] Master 127.29.243.254:41305 was elected leader, sending a full tablet report...
I20260809 04:59:14.914053 19430 tablet_bootstrap.cc:492] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3: Bootstrap starting.
I20260809 04:59:14.919651 19430 tablet_bootstrap.cc:654] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:14.921051 19430 log.cc:824] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:14.922353 19431 tablet_bootstrap.cc:492] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23: Bootstrap starting.
I20260809 04:59:14.926510 19431 tablet_bootstrap.cc:654] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:14.927819 19430 tablet_bootstrap.cc:492] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3: No bootstrap required, opened a new log
I20260809 04:59:14.927959 19431 log.cc:824] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:14.928151 19430 ts_tablet_manager.cc:1397] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.006s
I20260809 04:59:14.936049 19433 tablet_bootstrap.cc:492] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b: Bootstrap starting.
I20260809 04:59:14.937817 19431 tablet_bootstrap.cc:492] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23: No bootstrap required, opened a new log
I20260809 04:59:14.938110 19431 ts_tablet_manager.cc:1397] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.003s
I20260809 04:59:14.939121 19433 tablet_bootstrap.cc:654] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:14.940553 19433 log.cc:824] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:14.944880 19430 raft_consensus.cc:348] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1418ffc730534838a70243aa85b5107b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 44543 } } peers { permanent_uuid: "8e91c40086114bdfaefeef6cd27c3b23" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 46037 } } peers { permanent_uuid: "1bf6136286524284bb7e896ff04bbcd3" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46177 } }
I20260809 04:59:14.945433 19430 raft_consensus.cc:374] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:14.945603 19430 raft_consensus.cc:729] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bf6136286524284bb7e896ff04bbcd3, State: Initialized, Role: FOLLOWER
I20260809 04:59:14.946401 19430 consensus_queue.cc:260] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1418ffc730534838a70243aa85b5107b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 44543 } } peers { permanent_uuid: "8e91c40086114bdfaefeef6cd27c3b23" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 46037 } } peers { permanent_uuid: "1bf6136286524284bb7e896ff04bbcd3" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46177 } }
I20260809 04:59:14.948475 19431 raft_consensus.cc:348] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1418ffc730534838a70243aa85b5107b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 44543 } } peers { permanent_uuid: "8e91c40086114bdfaefeef6cd27c3b23" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 46037 } } peers { permanent_uuid: "1bf6136286524284bb7e896ff04bbcd3" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46177 } }
I20260809 04:59:14.948982 19431 raft_consensus.cc:374] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:14.949147 19431 raft_consensus.cc:729] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e91c40086114bdfaefeef6cd27c3b23, State: Initialized, Role: FOLLOWER
I20260809 04:59:14.949925 19431 consensus_queue.cc:260] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1418ffc730534838a70243aa85b5107b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 44543 } } peers { permanent_uuid: "8e91c40086114bdfaefeef6cd27c3b23" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 46037 } } peers { permanent_uuid: "1bf6136286524284bb7e896ff04bbcd3" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46177 } }
I20260809 04:59:14.954648 19430 ts_tablet_manager.cc:1428] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3: Time spent starting tablet: real 0.026s user 0.012s sys 0.008s
I20260809 04:59:14.956259 18943 heartbeater.cc:503] Master 127.29.243.254:41305 was elected leader, sending a full tablet report...
I20260809 04:59:14.956629 19431 ts_tablet_manager.cc:1428] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23: Time spent starting tablet: real 0.018s user 0.013s sys 0.005s
I20260809 04:59:14.960903 19433 tablet_bootstrap.cc:492] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b: No bootstrap required, opened a new log
I20260809 04:59:14.961200 19433 ts_tablet_manager.cc:1397] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b: Time spent bootstrapping tablet: real 0.025s user 0.012s sys 0.004s
I20260809 04:59:14.968353 19433 raft_consensus.cc:348] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1418ffc730534838a70243aa85b5107b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 44543 } } peers { permanent_uuid: "8e91c40086114bdfaefeef6cd27c3b23" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 46037 } } peers { permanent_uuid: "1bf6136286524284bb7e896ff04bbcd3" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46177 } }
I20260809 04:59:14.968708 19433 raft_consensus.cc:374] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:14.968801 19433 raft_consensus.cc:729] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1418ffc730534838a70243aa85b5107b, State: Initialized, Role: FOLLOWER
I20260809 04:59:14.969460 19433 consensus_queue.cc:260] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1418ffc730534838a70243aa85b5107b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 44543 } } peers { permanent_uuid: "8e91c40086114bdfaefeef6cd27c3b23" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 46037 } } peers { permanent_uuid: "1bf6136286524284bb7e896ff04bbcd3" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46177 } }
I20260809 04:59:14.970678 19379 heartbeater.cc:503] Master 127.29.243.254:41305 was elected leader, sending a full tablet report...
I20260809 04:59:14.971351 19433 ts_tablet_manager.cc:1428] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b: Time spent starting tablet: real 0.010s user 0.007s sys 0.005s
I20260809 04:59:15.011855 19439 raft_consensus.cc:482] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:59:15.012172 19439 raft_consensus.cc:504] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1418ffc730534838a70243aa85b5107b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 44543 } } peers { permanent_uuid: "8e91c40086114bdfaefeef6cd27c3b23" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 46037 } } peers { permanent_uuid: "1bf6136286524284bb7e896ff04bbcd3" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46177 } }
I20260809 04:59:15.014192 19439 leader_election.cc:288] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e91c40086114bdfaefeef6cd27c3b23 (127.29.243.193:46037), 1bf6136286524284bb7e896ff04bbcd3 (127.29.243.194:46177)
I20260809 04:59:15.020669 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.014386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:46037 (local address 127.29.243.195:44677)
0809 04:59:15.014566 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:15.014594 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:15.014718 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:15.015364 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:15.015369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:15.015403 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:15.015650 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:15.015662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.016985 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.016989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:15.018406 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:15.018425 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.019959 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.019965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:15.020138 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:15.020173 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 04:59:15.020265 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:59:15.020396 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 04:59:15.020679 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.014772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:44677 (local address 127.29.243.193:46037)
0809 04:59:15.014975 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:15.014981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:15.014999 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:59:15.015067 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:15.015072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:15.015154 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:15.015236 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:15.015922 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.016865 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.018581 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.019797 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.019969 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:15.020044 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:15.020314 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 04:59:15.020441 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:59:15.021687 19440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.014669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:46177 (local address 127.29.243.195:59077)
0809 04:59:15.015786 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:15.015808 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:15.015919 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:15.016269 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:15.016273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:15.016299 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:15.016628 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:15.016638 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.017961 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.017966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:15.019443 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:15.019455 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.020999 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.021004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:15.021163 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:15.021186 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:59:15.021279 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 04:59:15.021403 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"thread_start_us":911,"threads_started":1}
I20260809 04:59:15.022056 18856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061d9d7153e8479195e5828cb5d5510e" candidate_uuid: "1418ffc730534838a70243aa85b5107b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e91c40086114bdfaefeef6cd27c3b23" is_pre_election: true
I20260809 04:59:15.022579 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.015095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:59077 (local address 127.29.243.194:46177)
0809 04:59:15.015313 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:15.015319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:15.015890 (+ 571us) server_negotiation.cc:407] Connection header received
0809 04:59:15.016055 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:15.016060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:15.016147 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:15.016232 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:15.016765 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.017789 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.019595 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.020653 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.020809 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:15.020879 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:15.021309 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 04:59:15.021438 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:59:15.022706 18856 raft_consensus.cc:2454] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1418ffc730534838a70243aa85b5107b in term 0.
I20260809 04:59:15.023514 19213 leader_election.cc:302] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1418ffc730534838a70243aa85b5107b, 8e91c40086114bdfaefeef6cd27c3b23; no voters:
I20260809 04:59:15.023522 19108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061d9d7153e8479195e5828cb5d5510e" candidate_uuid: "1418ffc730534838a70243aa85b5107b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bf6136286524284bb7e896ff04bbcd3" is_pre_election: true
I20260809 04:59:15.023979 19439 raft_consensus.cc:2781] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:59:15.024032 19108 raft_consensus.cc:2454] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1418ffc730534838a70243aa85b5107b in term 0.
I20260809 04:59:15.024132 19439 raft_consensus.cc:482] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:59:15.024223 19439 raft_consensus.cc:3037] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:15.026734 19439 raft_consensus.cc:504] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1418ffc730534838a70243aa85b5107b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 44543 } } peers { permanent_uuid: "8e91c40086114bdfaefeef6cd27c3b23" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 46037 } } peers { permanent_uuid: "1bf6136286524284bb7e896ff04bbcd3" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46177 } }
I20260809 04:59:15.027338 19439 leader_election.cc:288] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [CANDIDATE]: Term 1 election: Requested vote from peers 8e91c40086114bdfaefeef6cd27c3b23 (127.29.243.193:46037), 1bf6136286524284bb7e896ff04bbcd3 (127.29.243.194:46177)
I20260809 04:59:15.027787 18856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061d9d7153e8479195e5828cb5d5510e" candidate_uuid: "1418ffc730534838a70243aa85b5107b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e91c40086114bdfaefeef6cd27c3b23"
I20260809 04:59:15.027973 18856 raft_consensus.cc:3037] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:15.027917 19108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061d9d7153e8479195e5828cb5d5510e" candidate_uuid: "1418ffc730534838a70243aa85b5107b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bf6136286524284bb7e896ff04bbcd3"
I20260809 04:59:15.028124 19108 raft_consensus.cc:3037] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:15.030292 18856 raft_consensus.cc:2454] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1418ffc730534838a70243aa85b5107b in term 1.
I20260809 04:59:15.030668 19213 leader_election.cc:302] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1418ffc730534838a70243aa85b5107b, 8e91c40086114bdfaefeef6cd27c3b23; no voters:
I20260809 04:59:15.030925 19439 raft_consensus.cc:2781] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:59:15.031414 19439 raft_consensus.cc:686] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 1 LEADER]: Becoming Leader. State: Replica: 1418ffc730534838a70243aa85b5107b, State: Running, Role: LEADER
I20260809 04:59:15.031764 19108 raft_consensus.cc:2454] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1418ffc730534838a70243aa85b5107b in term 1.
I20260809 04:59:15.032269 19439 consensus_queue.cc:237] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1418ffc730534838a70243aa85b5107b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 44543 } } peers { permanent_uuid: "8e91c40086114bdfaefeef6cd27c3b23" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 46037 } } peers { permanent_uuid: "1bf6136286524284bb7e896ff04bbcd3" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46177 } }
I20260809 04:59:15.037415 18610 catalog_manager.cc:5495] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1418ffc730534838a70243aa85b5107b (127.29.243.195). New cstate: current_term: 1 leader_uuid: "1418ffc730534838a70243aa85b5107b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1418ffc730534838a70243aa85b5107b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 44543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e91c40086114bdfaefeef6cd27c3b23" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bf6136286524284bb7e896ff04bbcd3" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260809 04:59:15.059214 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.053501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54278 (local address 127.29.243.195:44543)
0809 04:59:15.053704 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:15.053710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:15.053815 (+ 105us) server_negotiation.cc:407] Connection header received
0809 04:59:15.054039 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:15.054043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:15.054128 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:15.054265 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:59:15.054689 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.055436 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.056837 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.057238 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.057371 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:15.058005 (+ 634us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:59:15.058571 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:59:15.058690 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:15.058821 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 04:59:15.058892 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:59:15.067951 18856 raft_consensus.cc:1261] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23 [term 1 FOLLOWER]: Refusing update from remote peer 1418ffc730534838a70243aa85b5107b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:59:15.067960 19108 raft_consensus.cc:1261] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3 [term 1 FOLLOWER]: Refusing update from remote peer 1418ffc730534838a70243aa85b5107b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:59:15.069015 19439 consensus_queue.cc:1035] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e91c40086114bdfaefeef6cd27c3b23" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:59:15.069334 19441 consensus_queue.cc:1035] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bf6136286524284bb7e896ff04bbcd3" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:59:15.087303 19445 mvcc.cc:205] Tried to move back new op lower bound from 7316486369547083776 to 7316486369416458240. Current Snapshot: MvccSnapshot[applied={T|T < 7316486369547083776}]
I20260809 04:59:15.089133 19451 mvcc.cc:205] Tried to move back new op lower bound from 7316486369547083776 to 7316486369416458240. Current Snapshot: MvccSnapshot[applied={T|T < 7316486369547083776}]
I20260809 04:59:15.107411 18610 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:59:15.107731 18610 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:59:15.112071 18610 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 566beff32e9e415d9263edb9ef4573d9: Sending DeleteTablet for 3 replicas of tablet 061d9d7153e8479195e5828cb5d5510e
I20260809 04:59:15.113461 18819 tablet_service.cc:1514] Processing DeleteTablet for tablet 061d9d7153e8479195e5828cb5d5510e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:15 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:52910
I20260809 04:59:15.113999 19274 tablet_service.cc:1514] Processing DeleteTablet for tablet 061d9d7153e8479195e5828cb5d5510e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:15 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:54268
I20260809 04:59:15.119997 19077 tablet_service.cc:1514] Processing DeleteTablet for tablet 061d9d7153e8479195e5828cb5d5510e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:15 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:55984
I20260809 04:59:15.120771 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 18682
I20260809 04:59:15.134177 19465 tablet_replica.cc:330] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b: stopping tablet replica
I20260809 04:59:15.134680 19465 raft_consensus.cc:2229] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:59:15.135412 19465 raft_consensus.cc:2258] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:15.136261 19467 tablet_replica.cc:330] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3: stopping tablet replica
I20260809 04:59:15.136726 19467 raft_consensus.cc:2229] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:59:15.137253 19467 raft_consensus.cc:2258] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:15.139659 19465 ts_tablet_manager.cc:1905] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:15.140416 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.121765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52924 (local address 127.29.243.193:46037)
0809 04:59:15.122253 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:15.122259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:15.122277 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:59:15.122332 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:15.122337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:15.122418 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:15.122519 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:15.124775 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.125873 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.127929 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.128626 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.128744 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:15.132443 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:15.132461 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:15.132482 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:15.132526 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:15.136441 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:15.137337 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:15.137346 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:15.137349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:15.137517 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:15.137995 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:15.138000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:15.138003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:15.138459 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:15.138784 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:15.139750 (+ 966us) server_negotiation.cc:299] Negotiation successful
0809 04:59:15.139976 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357}
I20260809 04:59:15.141654 18932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:15.146962 19465 ts_tablet_manager.cc:1918] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:59:15.147135 19465 log.cc:1196] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/061d9d7153e8479195e5828cb5d5510e
I20260809 04:59:15.149354 19467 ts_tablet_manager.cc:1905] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:15.156570 19467 ts_tablet_manager.cc:1918] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:59:15.156773 19467 log.cc:1196] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/061d9d7153e8479195e5828cb5d5510e
I20260809 04:59:15.157303 19467 ts_tablet_manager.cc:1939] T 061d9d7153e8479195e5828cb5d5510e P 1bf6136286524284bb7e896ff04bbcd3: Deleting consensus metadata
I20260809 04:59:15.159379 18592 catalog_manager.cc:4841] TS 1bf6136286524284bb7e896ff04bbcd3 (127.29.243.194:46177): tablet 061d9d7153e8479195e5828cb5d5510e (table test-table [id=2026f7bf8a5e4211b167b7ca35eaa532]) successfully deleted
I20260809 04:59:15.160218 19465 ts_tablet_manager.cc:1939] T 061d9d7153e8479195e5828cb5d5510e P 1418ffc730534838a70243aa85b5107b: Deleting consensus metadata
I20260809 04:59:15.161970 18588 catalog_manager.cc:4841] TS 1418ffc730534838a70243aa85b5107b (127.29.243.195:44543): tablet 061d9d7153e8479195e5828cb5d5510e (table test-table [id=2026f7bf8a5e4211b167b7ca35eaa532]) successfully deleted
I20260809 04:59:15.315385 19464 tablet_replica.cc:330] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23: stopping tablet replica
I20260809 04:59:15.316076 19464 raft_consensus.cc:2229] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:59:15.316956 19464 raft_consensus.cc:2258] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:15.320187 19464 ts_tablet_manager.cc:1905] T 061d9d7153e8479195e5828cb5d5510e P 8e91c40086114bdfaefeef6cd27c3b23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:15.324947 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 18682
I20260809 04:59:15.360481 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 18948
W20260809 04:59:15.360760 18591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.243.193:46037 (error 108)
I20260809 04:59:15.374011 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.373551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:46037 (local address 127.0.0.1:52938)
0809 04:59:15.373702 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:15.373828 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 04:59:15.374413 18591 catalog_manager.cc:4584] TS 8e91c40086114bdfaefeef6cd27c3b23 (127.29.243.193:46037): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 061d9d7153e8479195e5828cb5d5510e: Network error: Client connection negotiation failed: client connection to 127.29.243.193:46037: connect: Connection refused (error 111)
I20260809 04:59:15.384371 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.361947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55992 (local address 127.29.243.194:46177)
0809 04:59:15.362572 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:15.362578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:15.362597 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:59:15.362661 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:15.362667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:15.362745 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:15.362839 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:15.364986 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.366104 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.368175 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.368818 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.368943 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:15.373096 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:15.373119 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:15.373143 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:15.373233 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:15.380160 (+ 6927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:15.381022 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:15.381033 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:15.381037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:15.381341 (+ 304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:15.381961 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:15.381966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:15.381969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:15.382565 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:15.382957 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:15.383201 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 04:59:15.383479 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490}
I20260809 04:59:15.385390 19172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:15.428421 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.427779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:46037 (local address 127.0.0.1:52948)
0809 04:59:15.427959 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:15.428158 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 04:59:15.429932 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.429423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:46037 (local address 127.0.0.1:52958)
0809 04:59:15.429560 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:15.429660 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 04:59:15.496552 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.495870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:46037 (local address 127.0.0.1:52966)
0809 04:59:15.496089 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:15.496202 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 04:59:15.498167 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.497730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:46037 (local address 127.0.0.1:52970)
0809 04:59:15.497871 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:15.497957 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 04:59:15.535729 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 18948
I20260809 04:59:15.573686 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 19182
I20260809 04:59:15.579015 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.578501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:46037 (local address 127.0.0.1:52976)
0809 04:59:15.578734 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:15.578838 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 04:59:15.580106 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.579739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:46037 (local address 127.0.0.1:52978)
0809 04:59:15.579929 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:15.579997 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 04:59:15.593200 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.574738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54292 (local address 127.29.243.195:44543)
0809 04:59:15.575390 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:15.575394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:15.575411 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:59:15.575468 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:15.575471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:15.575538 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:15.575623 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:15.577759 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.578727 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.580830 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:15.581323 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:15.581430 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:15.585194 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:15.585208 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:15.585227 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:15.585269 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:15.589741 (+ 4472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:15.590687 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:15.590697 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:15.590700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:15.590877 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:15.591411 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:15.591417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:15.591420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:15.591825 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:15.592150 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:15.592549 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 04:59:15.592784 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":346,"threads_started":1}
I20260809 04:59:15.594301 19373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:15.716455 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 19182
I20260809 04:59:15.737660 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 18562
I20260809 04:59:15.739028 18670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:15.851495 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:15.850861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:46037 (local address 127.0.0.1:52992)
0809 04:59:15.851107 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:15.851214 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 04:59:15.851626 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 18562
I20260809 04:59:15.885918 30671 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 04:59:16.223057 30671 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 04:59:16.223300 30671 mini_ranger.cc:66] Stopping Ranger...
I20260809 04:59:16.568800 30671 mini_ranger.cc:68] Stopped Ranger
2026-08-09 04:59:16.569 UTC [17070] LOG: received smart shutdown request
2026-08-09 04:59:16.571 UTC [17070] LOG: background worker "logical replication launcher" (PID 17083) exited with exit code 1
2026-08-09 04:59:16.571 UTC [17076] LOG: shutting down
2026-08-09 04:59:16.730 UTC [17070] LOG: database system is shut down
2026-08-09T04:59:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51624 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:59:16 dist-test-slave-21b9 krb5kdc[19487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:59:16 dist-test-slave-21b9 krb5kdc[19487](info): set up 2 sockets
Aug 09 04:59:16 dist-test-slave-21b9 krb5kdc[19487](info): commencing operation
krb5kdc: starting...
W20260809 04:59:18.099350 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:59:18 dist-test-slave-21b9 krb5kdc[19487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:59:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:59:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:59:18.373028 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41499
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41499,127.29.243.253:46869,127.29.243.252:41445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:18.715960 19508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:18.716264 19508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:18.716354 19508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:18.723541 19508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:59:18.723651 19508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:18.723699 19508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:18.723757 19508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:59:18.723801 19508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:59:18.733566 19508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41499,127.29.243.253:46869,127.29.243.252:41445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:18.734882 19508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:18.736215 19508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:18.747776 19513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:18.747738 19514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:18.748025 19516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:18.748764 19508 server_base.cc:1034] running on GCE node
I20260809 04:59:18.749560 19508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:18.750949 19508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:18.752164 19508 hybrid_clock.cc:648] HybridClock initialized: now 1786251558752113 us; error 44 us; skew 500 ppm
I20260809 04:59:18.752534 19508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:18 dist-test-slave-21b9 krb5kdc[19487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251558, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:18.758883 19508 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:59:18.760387 19508 webserver.cc:460] Webserver started at http://127.29.243.254:46859/ using document root <none> and password file <none>
I20260809 04:59:18.761085 19508 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:18.761195 19508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:18.761461 19508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:18.764518 19508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f"
format_stamp: "Formatted at 2026-08-09 04:59:18 on dist-test-slave-21b9"
server_key: "bd4696302cda485a2ebfab8634714855"
server_key_iv: "55ee93442fbaff67c80dc0169f127214"
server_key_version: "encryptionkey@0"
I20260809 04:59:18.765312 19508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f"
format_stamp: "Formatted at 2026-08-09 04:59:18 on dist-test-slave-21b9"
server_key: "bd4696302cda485a2ebfab8634714855"
server_key_iv: "55ee93442fbaff67c80dc0169f127214"
server_key_version: "encryptionkey@0"
I20260809 04:59:18.770609 19508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:59:18.774060 19527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:18.775396 19508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:59:18.775547 19508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f"
format_stamp: "Formatted at 2026-08-09 04:59:18 on dist-test-slave-21b9"
server_key: "bd4696302cda485a2ebfab8634714855"
server_key_iv: "55ee93442fbaff67c80dc0169f127214"
server_key_version: "encryptionkey@0"
I20260809 04:59:18.775714 19508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:18.790910 19508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:18.794983 19508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:18.795218 19508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:18.813448 19508 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:41499
I20260809 04:59:18.813479 19603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:41499 every 8 connection(s)
I20260809 04:59:18.814818 19508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:59:18.818622 19604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:18.818766 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 19508
I20260809 04:59:18.818984 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:59:18.819693 30671 external_mini_cluster.cc:1428] Setting key 976cbc1a06f06270049581ac1e5b627f
I20260809 04:59:18.825286 19604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:59:18 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251558, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.243.253@KRBTEST.COM; defaulting to no policy
I20260809 04:59:18.849180 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:18.828485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41499 (local address 127.0.0.1:49304)
0809 04:59:18.829499 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:18.829551 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:18.831932 (+ 2381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:18.832893 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:18.832899 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:18.833448 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:18.833790 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:18.833807 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:18.835886 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:18.835890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:18.836940 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:18.836952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:18.838126 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:18.838131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:18.838251 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:18.838847 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:18.838869 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:18.841009 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:18.844936 (+ 3927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:18.844947 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:18.844962 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:18.845346 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:18.845840 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:18.845843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:18.845845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:18.846006 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:18.847048 (+ 1042us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:18.847056 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:18.847384 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 04:59:18.847626 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 04:59:18.848141 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"mutex_wait_us":126,"thread_start_us":477,"threads_started":1}
I20260809 04:59:18.849216 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:18.828485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49304 (local address 127.29.243.254:41499)
0809 04:59:18.830422 (+ 1937us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:18.830429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:18.830477 (+ 48us) server_negotiation.cc:407] Connection header received
0809 04:59:18.832314 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:18.832322 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:18.832651 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:18.832850 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:18.833985 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:18.835732 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:18.837089 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:18.837896 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:18.838092 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:18.841244 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:18.841255 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:18.841272 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:18.841307 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:18.844805 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:18.845480 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:18.845490 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:18.845493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:18.845717 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:18.846116 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:18.846121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:18.846124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:18.846828 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:18.847050 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:18.847762 (+ 712us) server_negotiation.cc:299] Negotiation successful
0809 04:59:18.848027 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1747,"spinlock_wait_cycles":39296,"thread_start_us":1488,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.253@KRBTEST.COM" created.
I20260809 04:59:18.852293 19604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:59:18.853281 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:18.852917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54080)
0809 04:59:18.853067 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:18.853173 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 04:59:18.853529 19531 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
I20260809 04:59:18.855072 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:18.854746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54082)
0809 04:59:18.854881 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:18.854949 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:59:18.855263 19604 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:46869: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab.
Entry for principal kudu/127.29.243.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab.
I20260809 04:59:18.882405 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.253:46869
--webserver_interface=127.29.243.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab
--principal=kudu/127.29.243.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41499,127.29.243.253:46869,127.29.243.252:41445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:59:18.904700 19604 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } attempt: 1
I20260809 04:59:18.905897 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:18.905430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54096)
0809 04:59:18.905633 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:18.905757 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 04:59:18.906957 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:18.906600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54102)
0809 04:59:18.906744 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:18.906816 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 04:59:18.907130 19604 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:46869: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
I20260809 04:59:18.975394 19604 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } attempt: 2
I20260809 04:59:18.976683 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:18.976149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54118)
0809 04:59:18.976423 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:18.976529 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 04:59:18.977895 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:18.977485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54124)
0809 04:59:18.977661 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:18.977738 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 04:59:18.978106 19604 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:46869: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
I20260809 04:59:19.069367 19604 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } attempt: 3
I20260809 04:59:19.070593 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.070145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54128)
0809 04:59:19.070342 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.070461 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 04:59:19.071775 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.071336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54144)
0809 04:59:19.071480 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.071558 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 04:59:19.072016 19604 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:46869: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
I20260809 04:59:19.215294 19604 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } attempt: 4
I20260809 04:59:19.216974 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.216063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54154)
0809 04:59:19.216658 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.216780 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397}
I20260809 04:59:19.218382 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.217898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54160)
0809 04:59:19.218086 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.218210 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 04:59:19.218719 19604 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:46869: Network error: Client connection negotiation failed: client connection to 127.29.243.253:46869: connect: Connection refused (error 111)
W20260809 04:59:19.292167 19614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:19.292480 19614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:19.292611 19614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:19.299767 19614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:59:19.299881 19614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:19.299969 19614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:19.300061 19614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:59:19.300120 19614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:59:19.310038 19614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41499,127.29.243.253:46869,127.29.243.252:41445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab
--principal=kudu/127.29.243.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.253:46869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.243.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:19.311407 19614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:19.312695 19614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:19.324419 19628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:19.324587 19624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:19.324563 19623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:19.325537 19614 server_base.cc:1034] running on GCE node
I20260809 04:59:19.326344 19614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:19.327798 19614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:19.329011 19614 hybrid_clock.cc:648] HybridClock initialized: now 1786251559328931 us; error 57 us; skew 500 ppm
I20260809 04:59:19.329375 19614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251559, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:19.332432 19614 init.cc:377] Logged in from keytab as kudu/127.29.243.253@KRBTEST.COM (short username kudu)
I20260809 04:59:19.334026 19614 webserver.cc:460] Webserver started at http://127.29.243.253:33361/ using document root <none> and password file <none>
I20260809 04:59:19.334781 19614 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:19.334903 19614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:19.335202 19614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:19.338212 19614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/data/instance:
uuid: "0168cb199e774099a22d75f406ae68fb"
format_stamp: "Formatted at 2026-08-09 04:59:19 on dist-test-slave-21b9"
server_key: "3e6057c4100a4e3d4abab6ff258be39f"
server_key_iv: "5c352ec456f2b5342995aad7d8cd5872"
server_key_version: "encryptionkey@0"
I20260809 04:59:19.339002 19614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/wal/instance:
uuid: "0168cb199e774099a22d75f406ae68fb"
format_stamp: "Formatted at 2026-08-09 04:59:19 on dist-test-slave-21b9"
server_key: "3e6057c4100a4e3d4abab6ff258be39f"
server_key_iv: "5c352ec456f2b5342995aad7d8cd5872"
server_key_version: "encryptionkey@0"
I20260809 04:59:19.344250 19614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 04:59:19.347723 19643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:19.349026 19614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:59:19.349200 19614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/wal
uuid: "0168cb199e774099a22d75f406ae68fb"
format_stamp: "Formatted at 2026-08-09 04:59:19 on dist-test-slave-21b9"
server_key: "3e6057c4100a4e3d4abab6ff258be39f"
server_key_iv: "5c352ec456f2b5342995aad7d8cd5872"
server_key_version: "encryptionkey@0"
I20260809 04:59:19.349382 19614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:19.364840 19614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:19.369288 19614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:19.369613 19614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:19.388024 19614 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.253:46869
I20260809 04:59:19.388048 19714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.253:46869 every 8 connection(s)
I20260809 04:59:19.389398 19614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
I20260809 04:59:19.390481 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 19614
I20260809 04:59:19.390683 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-1/wal/instance
I20260809 04:59:19.391285 30671 external_mini_cluster.cc:1428] Setting key 144a7dee3a20641760909cd50fa1c9b5
I20260809 04:59:19.394325 19715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:19.402993 19715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251559, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.243.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.252@KRBTEST.COM" created.
I20260809 04:59:19.425784 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.405756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.29.243.254:41499)
0809 04:59:19.406286 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:19.406293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:19.407432 (+ 1139us) server_negotiation.cc:407] Connection header received
0809 04:59:19.409883 (+ 2451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:19.409887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:19.409967 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:19.410063 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:19.412181 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.412885 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.414994 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.415438 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.415542 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.419549 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:19.419556 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:19.419561 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:19.419634 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:19.422532 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.423387 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.423391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.423393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.423569 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.424038 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.424042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.424044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.424317 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:19.424446 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:19.425285 (+ 839us) server_negotiation.cc:299] Negotiation successful
0809 04:59:19.425489 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":270,"threads_started":1}
I20260809 04:59:19.427073 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.406384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41499 (local address 127.0.0.1:49308)
0809 04:59:19.407207 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.407254 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:19.409593 (+ 2339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:19.410363 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:19.410374 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:19.411075 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:19.411952 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.411976 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.413054 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.413059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.414858 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.414873 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.415680 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.415687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.415956 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.416674 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:19.416703 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:19.419250 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:19.422668 (+ 3418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.422676 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.422691 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.423226 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.423692 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.423695 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.423698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.423903 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.424455 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:19.424463 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:19.424892 (+ 429us) client_negotiation.cc:769] Sending connection context
0809 04:59:19.425213 (+ 321us) client_negotiation.cc:240] Negotiation successful
0809 04:59:19.425734 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":316,"threads_started":1}
I20260809 04:59:19.428855 19715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251559, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
I20260809 04:59:19.443617 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.429532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54168)
0809 04:59:19.429700 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.429718 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:19.429804 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:19.430909 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:19.430917 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:19.431384 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:19.431561 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.431572 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.433009 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.433014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.434112 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.434128 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.434953 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.434959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.435168 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.435746 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:19.435763 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:19.437878 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:19.440957 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.440962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.440966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.441421 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.441779 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.441784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.441787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.441987 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.442572 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:19.442576 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:19.442731 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 04:59:19.442868 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 04:59:19.443165 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 04:59:19.443647 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.429840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.29.243.253:46869)
0809 04:59:19.430510 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:19.430516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:19.430541 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:59:19.430582 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:19.430588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:19.430755 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:19.430867 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:19.431793 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.432848 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.434252 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.434792 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.434887 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.438090 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:19.438102 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:19.438116 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:19.438146 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:19.440817 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.441546 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.441553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.441555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.441677 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.442088 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.442091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.442093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.442434 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:19.442594 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:19.443003 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 04:59:19.443263 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":513,"thread_start_us":376,"threads_started":1}
I20260809 04:59:19.446333 19715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:59:19.447582 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.447013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56140)
0809 04:59:19.447205 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.447379 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 04:59:19.447826 19647 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
I20260809 04:59:19.449440 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.449018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56154)
0809 04:59:19.449211 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.449309 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 04:59:19.449708 19715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:41445: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab.
Entry for principal kudu/127.29.243.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab.
I20260809 04:59:19.458597 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.252:41445
--webserver_interface=127.29.243.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab
--principal=kudu/127.29.243.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41499,127.29.243.253:46869,127.29.243.252:41445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:59:19.499032 19715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } attempt: 1
I20260809 04:59:19.500411 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.499792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56158)
0809 04:59:19.500046 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.500190 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 04:59:19.501509 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.501103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56164)
0809 04:59:19.501274 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.501367 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:59:19.501907 19715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:41445: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
I20260809 04:59:19.518016 19604 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } attempt: 5
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251558, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
I20260809 04:59:19.531481 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.519061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54182 (local address 127.29.243.253:46869)
0809 04:59:19.519375 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:19.519380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:19.519401 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:59:19.519482 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:19.519486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:19.519551 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:19.519637 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:19.520430 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.521110 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.522272 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.522745 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.522867 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.526202 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:19.526207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:19.526210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:19.526243 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:19.528877 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.529693 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.529697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.529700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.529818 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.530211 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.530213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.530215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.530482 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:19.530588 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:19.531024 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 04:59:19.531187 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260809 04:59:19.531858 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.518841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54182)
0809 04:59:19.519156 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.519178 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:19.519343 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:19.519694 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:19.519699 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:19.520078 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:19.520270 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.520279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.521242 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.521246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.522135 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.522148 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.522894 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.522900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.523101 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.523748 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:19.523767 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:19.526012 (+ 2245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:19.529031 (+ 3019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.529037 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.529041 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.529584 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.529929 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.529934 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.529937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.530118 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.530592 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:19.530597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:19.530781 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 04:59:19.530952 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 04:59:19.531327 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260809 04:59:19.533030 19604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:59:19.534327 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.533879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56174)
0809 04:59:19.534071 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.534167 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 04:59:19.535427 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.535024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56176)
0809 04:59:19.535163 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.535263 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 04:59:19.535634 19604 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:41445: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
I20260809 04:59:19.570152 19715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } attempt: 2
I20260809 04:59:19.571455 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.570854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56190)
0809 04:59:19.571073 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.571196 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 04:59:19.572531 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.572128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56202)
0809 04:59:19.572290 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.572385 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 04:59:19.572772 19715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:41445: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
I20260809 04:59:19.586885 19604 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } attempt: 1
I20260809 04:59:19.588061 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.587529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56212)
0809 04:59:19.587733 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.587857 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 04:59:19.589035 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.588667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56228)
0809 04:59:19.588804 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.588898 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 04:59:19.589310 19604 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:41445: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
I20260809 04:59:19.657552 19604 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } attempt: 2
I20260809 04:59:19.658953 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.658299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56234)
0809 04:59:19.658575 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.658704 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 04:59:19.660219 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.659805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56250)
0809 04:59:19.659973 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.660072 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 04:59:19.660481 19604 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:41445: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
I20260809 04:59:19.663996 19715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } attempt: 3
I20260809 04:59:19.665077 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.664557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56258)
0809 04:59:19.664761 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.664889 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 04:59:19.666209 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.665688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56268)
0809 04:59:19.665955 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.666056 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260809 04:59:19.666452 19715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:41445: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
I20260809 04:59:19.766729 19604 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } attempt: 3
I20260809 04:59:19.768028 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.767473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56274)
0809 04:59:19.767701 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.767814 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 04:59:19.769109 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.768653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56290)
0809 04:59:19.768865 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.768961 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260809 04:59:19.769301 19604 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:41445: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
I20260809 04:59:19.809682 19715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } attempt: 4
I20260809 04:59:19.810969 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.810370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56296)
0809 04:59:19.810645 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.810753 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 04:59:19.811976 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.811627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56312)
0809 04:59:19.811761 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.811850 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 04:59:19.812191 19715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:41445: Network error: Client connection negotiation failed: client connection to 127.29.243.252:41445: connect: Connection refused (error 111)
W20260809 04:59:19.813764 19732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:19.814021 19732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:19.814193 19732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:19.821385 19732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:59:19.821487 19732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:19.821559 19732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:19.821638 19732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:59:19.821696 19732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:59:19.831740 19732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41499,127.29.243.253:46869,127.29.243.252:41445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab
--principal=kudu/127.29.243.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.252:41445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.243.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:19.833078 19732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:19.834411 19732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:19.846088 19739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:19.846276 19740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:19.846434 19743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:19.847018 19732 server_base.cc:1034] running on GCE node
I20260809 04:59:19.847805 19732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:19.849202 19732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:19.850412 19732 hybrid_clock.cc:648] HybridClock initialized: now 1786251559850364 us; error 41 us; skew 500 ppm
I20260809 04:59:19.850728 19732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251559, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:19.853884 19732 init.cc:377] Logged in from keytab as kudu/127.29.243.252@KRBTEST.COM (short username kudu)
I20260809 04:59:19.855397 19732 webserver.cc:460] Webserver started at http://127.29.243.252:42533/ using document root <none> and password file <none>
I20260809 04:59:19.856081 19732 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:19.856189 19732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:19.856462 19732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:19.859491 19732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/data/instance:
uuid: "d410fd0584314486b989d4ded693bd2d"
format_stamp: "Formatted at 2026-08-09 04:59:19 on dist-test-slave-21b9"
server_key: "7a1b8895944777a77cbddfe106d6c201"
server_key_iv: "eb72e00a7d2d62614a4cedd9287ae053"
server_key_version: "encryptionkey@0"
I20260809 04:59:19.860330 19732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/wal/instance:
uuid: "d410fd0584314486b989d4ded693bd2d"
format_stamp: "Formatted at 2026-08-09 04:59:19 on dist-test-slave-21b9"
server_key: "7a1b8895944777a77cbddfe106d6c201"
server_key_iv: "eb72e00a7d2d62614a4cedd9287ae053"
server_key_version: "encryptionkey@0"
I20260809 04:59:19.865548 19732 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:59:19.869122 19754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:19.870431 19732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:59:19.870590 19732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/wal
uuid: "d410fd0584314486b989d4ded693bd2d"
format_stamp: "Formatted at 2026-08-09 04:59:19 on dist-test-slave-21b9"
server_key: "7a1b8895944777a77cbddfe106d6c201"
server_key_iv: "eb72e00a7d2d62614a4cedd9287ae053"
server_key_version: "encryptionkey@0"
I20260809 04:59:19.870776 19732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:19.903327 19732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:19.907490 19732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:19.907766 19732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:19.925397 19732 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.252:41445
I20260809 04:59:19.925447 19845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.252:41445 every 8 connection(s)
I20260809 04:59:19.926842 19732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
I20260809 04:59:19.930799 19847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:19.936256 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 19732
I20260809 04:59:19.936422 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786251347713215-30671-0/minicluster-data/master-2/wal/instance
I20260809 04:59:19.936880 30671 external_mini_cluster.cc:1428] Setting key 5031a2bfbe6d5d8d5697f5cb2cfce82b
I20260809 04:59:19.937462 19847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:59:19.946542 19604 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } attempt: 4
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251559, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251558, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
I20260809 04:59:19.959026 19854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.940570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.29.243.254:41499)
0809 04:59:19.941056 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:19.941062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:19.941632 (+ 570us) server_negotiation.cc:407] Connection header received
0809 04:59:19.942271 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:19.942275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:19.942349 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:19.942435 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:19.943855 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.944713 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.946388 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.947029 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.947161 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.951401 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:19.951407 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:19.951410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:19.951453 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:19.955243 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.955980 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.955985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.955989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.956156 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.956731 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.956737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.956739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.957107 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:19.957261 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:19.958410 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0809 04:59:19.958647 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":228,"threads_started":1}
I20260809 04:59:19.960312 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.940572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41499 (local address 127.0.0.1:49316)
0809 04:59:19.941443 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.941517 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:19.942016 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:19.942535 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:19.942542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:19.943255 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:19.943684 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.943702 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.944878 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.944884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.946237 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.946250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.947473 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.947482 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.947779 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.948481 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:19.948511 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:19.951199 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:19.955384 (+ 4185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.955395 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.955407 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.955865 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.956282 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.956286 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.956290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.956593 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.957263 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:19.957271 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:19.957726 (+ 455us) client_negotiation.cc:769] Sending connection context
0809 04:59:19.958308 (+ 582us) client_negotiation.cc:240] Negotiation successful
0809 04:59:19.958834 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":442,"threads_started":1}
I20260809 04:59:19.961531 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.938778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.29.243.252:41445)
0809 04:59:19.939638 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:19.939652 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:19.939698 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:59:19.942016 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:19.942045 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:19.942334 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:19.942514 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:19.943977 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.945387 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.946950 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.947801 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.947931 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.953555 (+ 5624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:19.953569 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:19.953586 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:19.953629 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:19.957749 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.958505 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.958513 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.958517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.958685 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.959329 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.959334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.959337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.959726 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:19.960024 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:19.960381 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 04:59:19.961097 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":379,"threads_started":1}
I20260809 04:59:19.962234 19847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:59:19.962927 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.947085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56332)
0809 04:59:19.947350 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.947368 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:19.947465 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:19.948570 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:19.948575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:19.949053 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:19.949274 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.949285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.950648 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.950654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.951956 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.951974 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.952825 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.952832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.953035 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.953729 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:19.953751 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:19.955981 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:19.959797 (+ 3816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.959803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.959807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.960254 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.960669 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.960674 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.960678 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.960854 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.962063 (+ 1209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:19.962066 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:19.962188 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 04:59:19.962307 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 04:59:19.962559 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260809 04:59:19.963027 19858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.947132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56332 (local address 127.29.243.252:41445)
0809 04:59:19.948003 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:19.948008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:19.948038 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:59:19.948083 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:19.948088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:19.948411 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:19.948519 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:19.949433 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.950495 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.952135 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.952608 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.952789 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.956190 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:19.956195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:19.956198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:19.956230 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:19.959661 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.960392 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.960397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.960400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.960544 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.960994 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.960999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.961002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.961328 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:19.961556 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:19.962447 (+ 891us) server_negotiation.cc:299] Negotiation successful
0809 04:59:19.962651 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":542,"threads_started":1}
I20260809 04:59:19.969354 19604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f: Bootstrap starting.
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251559, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
I20260809 04:59:19.973443 19604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f: Neither blocks nor log segments found. Creating new log.
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:19.974784 19604 log.cc:824] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:19.976758 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.962868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:46869 (local address 127.0.0.1:54192)
0809 04:59:19.963434 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.963451 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:19.963550 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:19.964082 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:19.964087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:19.964414 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:19.964597 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.964609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.965811 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.965816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.966651 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.966662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.967545 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.967551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.967756 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.968331 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:19.968353 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:19.970742 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:19.973836 (+ 3094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.973841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.973845 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.974316 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.974780 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.974786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.974790 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.974939 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.975796 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:19.975801 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:19.975969 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 04:59:19.976136 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 04:59:19.976434 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456}
I20260809 04:59:19.977706 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.963210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.29.243.253:46869)
0809 04:59:19.963414 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:19.963421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:19.963498 (+ 77us) server_negotiation.cc:407] Connection header received
0809 04:59:19.963710 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:19.963715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:19.963801 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:19.963906 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:19.964727 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.965681 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.966783 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.967396 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.967522 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.970920 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:19.970925 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:19.970928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:19.970962 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:19.973696 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.974444 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.974449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.974452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.974669 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.975045 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.975050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.975054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.975528 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:19.975731 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:19.977118 (+ 1387us) server_negotiation.cc:299] Negotiation successful
0809 04:59:19.977342 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:59:19.977862 19604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f: No bootstrap required, opened a new log
I20260809 04:59:19.979007 19847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:59:19.981830 19854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.965072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.29.243.254:41499)
0809 04:59:19.965228 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:19.965233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:19.965250 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:59:19.965293 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:19.965297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:19.965376 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:19.965475 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:19.966957 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.967921 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.969213 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.969673 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.969806 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.974587 (+ 4781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:19.974592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:19.974597 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:19.974633 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:19.978106 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.978905 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.978910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.978913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.979060 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.980278 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.980283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.980287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.980658 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:19.980960 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:19.981277 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 04:59:19.981517 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251559, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
I20260809 04:59:19.988072 19604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } }
I20260809 04:59:19.988463 19604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:19.988592 19604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b2ec2eccc34b4aa76ca7709bf08f3f, State: Initialized, Role: FOLLOWER
I20260809 04:59:19.989319 19604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } }
I20260809 04:59:19.990343 19862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } } }
I20260809 04:59:19.990850 19862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:59:19.991771 19604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 04:59:19 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
I20260809 04:59:19.993744 19858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.980146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56336 (local address 127.29.243.252:41445)
0809 04:59:19.980304 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:19.980308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:19.980320 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:59:19.980352 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:19.980354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:19.980415 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:19.980489 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:19.981292 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.982239 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.983655 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.984314 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.984445 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.988030 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:19.988036 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:19.988040 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:19.988078 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:19.991062 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.991695 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.991700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.991704 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.991825 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.992174 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.992179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.992183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.992540 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:19.992686 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:19.993129 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 04:59:19.993369 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:59:19.994611 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.979924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56336)
0809 04:59:19.980082 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:19.980096 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:19.980168 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:19.980528 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:19.980532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:19.980944 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:19.981143 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.981156 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.982388 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.982392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.983496 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:19.983511 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.984601 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.984606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:19.984772 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.985312 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:19.985333 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:19.987840 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:19.991187 (+ 3347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.991192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.991196 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.991589 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.991940 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:19.991945 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:19.991949 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:19.992064 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:19.992678 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:19.992684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:19.992836 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 04:59:19.993967 (+ 1131us) client_negotiation.cc:240] Negotiation successful
0809 04:59:19.994268 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 04:59:20.001070 19847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d: Bootstrap starting.
I20260809 04:59:20.001121 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:19.984000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.29.243.253:46869)
0809 04:59:19.984171 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:19.984175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:19.984188 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:59:19.984225 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:19.984228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:19.984277 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:19.984352 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:19.985515 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.986217 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.987539 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:19.987980 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:19.988110 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:19.994065 (+ 5955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:19.994072 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:19.994076 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:19.994119 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:19.997340 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.998165 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.998170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.998173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.998333 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:19.998793 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:19.998799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:19.998802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:19.999147 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:19.999433 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:20.000509 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0809 04:59:20.000757 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:59:20.005239 19847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:20.006606 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:59:20.006616 19847 log.cc:824] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:20.002117 19508 master_runner.cc:186] Error getting master registration for 127.29.243.253:46869: OK, instance_id { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" instance_seqno: 1786251559382615 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:59:20.012609 19508 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:59:20.013279 19847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d: No bootstrap required, opened a new log
I20260809 04:59:20.021519 19847 raft_consensus.cc:348] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } }
I20260809 04:59:20.021855 19847 raft_consensus.cc:374] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:20.021975 19847 raft_consensus.cc:729] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d410fd0584314486b989d4ded693bd2d, State: Initialized, Role: FOLLOWER
I20260809 04:59:20.022815 19847 consensus_queue.cc:260] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } }
W20260809 04:59:20.025095 19882 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:59:20.025241 19882 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:59:20.026192 19889 sys_catalog.cc:455] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } } }
I20260809 04:59:20.026630 19889 sys_catalog.cc:458] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:59:20.027616 19847 sys_catalog.cc:564] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:59:20.048143 19537 tablet.cc:2369] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:59:20.068965 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:20.041720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54204 (local address 127.29.243.253:46869)
0809 04:59:20.041919 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:20.041926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:20.050300 (+ 8374us) server_negotiation.cc:407] Connection header received
0809 04:59:20.050442 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:20.050448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:20.050539 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:20.050631 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:20.053133 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:20.054183 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:20.056140 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:20.056782 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:20.056880 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:20.060502 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:20.060509 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:20.060513 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:20.060563 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:20.064094 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:20.066416 (+ 2322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:20.066422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:20.066426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:20.066592 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:20.067070 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:20.067076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:20.067080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:20.067500 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:20.067730 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:20.068325 (+ 595us) server_negotiation.cc:299] Negotiation successful
0809 04:59:20.068571 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:59:20.074486 19858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:20.028408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.29.243.252:41445)
0809 04:59:20.028600 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:20.028606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:20.046669 (+ 18063us) server_negotiation.cc:407] Connection header received
0809 04:59:20.047108 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:20.047115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:20.047215 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:20.047325 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:20.054104 (+ 6779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:20.055215 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:20.060194 (+ 4979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:20.060928 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:20.061028 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:20.064629 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:20.064636 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:20.064641 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:20.064684 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:20.068342 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:20.071950 (+ 3608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:20.071955 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:20.071958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:20.072126 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:20.072613 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:20.072618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:20.072621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:20.072984 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:20.073244 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:20.073873 (+ 629us) server_negotiation.cc:299] Negotiation successful
0809 04:59:20.074094 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:59:20.077620 19854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:20.028760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49336 (local address 127.29.243.254:41499)
0809 04:59:20.028936 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:20.028946 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:20.040932 (+ 11986us) server_negotiation.cc:407] Connection header received
0809 04:59:20.042699 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:20.042707 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:20.042806 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:20.042890 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:20.050824 (+ 7934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:20.052085 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:20.057482 (+ 5397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:20.058464 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:20.058577 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:20.069542 (+ 10965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:20.069550 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:20.069554 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:20.069599 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:20.073329 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:20.074204 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:20.074210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:20.074214 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:20.074364 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:20.074861 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:20.074866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:20.074873 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:20.075560 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:20.077040 (+ 1480us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:20.077155 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 04:59:20.077342 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 04:59:20.088924 19915 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:59:20.089093 19915 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:59:20.086825 19732 master_runner.cc:186] Error getting master registration for 127.29.243.253:46869: OK, instance_id { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" instance_seqno: 1786251559382615 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:59:20.098088 19732 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:59:20.111402 19715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } attempt: 5
Aug 09 04:59:20 dist-test-slave-21b9 krb5kdc[19487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251559, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
I20260809 04:59:20.129631 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:20.112061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:41445 (local address 127.0.0.1:56356)
0809 04:59:20.112325 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:20.112345 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:20.112455 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:20.112790 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:20.112795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:20.113354 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:20.113569 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:20.113582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:20.115100 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:20.115106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:20.116634 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:20.116650 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:20.117672 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:20.117679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:20.117933 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:20.119801 (+ 1868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:20.119826 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:20.122698 (+ 2872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:20.126682 (+ 3984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:20.126688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:20.126692 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:20.127191 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:20.127639 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:20.127645 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:20.127669 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:20.127872 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:20.128540 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:20.128546 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:20.128716 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 04:59:20.128860 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 04:59:20.129224 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260809 04:59:20.129652 19858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:20.112253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56356 (local address 127.29.243.252:41445)
0809 04:59:20.112438 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:20.112444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:20.112463 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:59:20.112571 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:20.112576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:20.112652 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:20.112751 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:20.113727 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:20.114946 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:20.116795 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:20.117515 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:20.117641 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:20.122911 (+ 5270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:20.122918 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:20.122923 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:20.122967 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:20.126537 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:20.127342 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:20.127348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:20.127351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:20.127518 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:20.128003 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:20.128008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:20.128012 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:20.128392 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:20.128566 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:20.129028 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 04:59:20.129277 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:59:20.136219 19715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb: Bootstrap starting.
I20260809 04:59:20.141626 19715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:20.143025 19715 log.cc:824] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:20.146426 19715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb: No bootstrap required, opened a new log
I20260809 04:59:20.157768 19715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } }
I20260809 04:59:20.158227 19715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:20.158365 19715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0168cb199e774099a22d75f406ae68fb, State: Initialized, Role: FOLLOWER
I20260809 04:59:20.159082 19715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } }
W20260809 04:59:20.159994 19762 tablet.cc:2369] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:59:20.161386 19922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } } }
I20260809 04:59:20.161868 19922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:59:20.162824 19715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:59:20.182560 19614 master_runner.cc:263] No leader master found from master 0168cb199e774099a22d75f406ae68fb
I20260809 04:59:20.182727 19614 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 04:59:20.186707 19942 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:59:20.186949 19942 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:59:20.220294 19862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:59:20.220628 19862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } }
I20260809 04:59:20.222388 19862 leader_election.cc:288] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0168cb199e774099a22d75f406ae68fb (127.29.243.253:46869), d410fd0584314486b989d4ded693bd2d (127.29.243.252:41445)
I20260809 04:59:20.223572 19794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d410fd0584314486b989d4ded693bd2d" is_pre_election: true
I20260809 04:59:20.224294 19794 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44b2ec2eccc34b4aa76ca7709bf08f3f in term 0.
I20260809 04:59:20.225185 19532 leader_election.cc:302] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44b2ec2eccc34b4aa76ca7709bf08f3f, d410fd0584314486b989d4ded693bd2d; no voters:
I20260809 04:59:20.226598 19682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0168cb199e774099a22d75f406ae68fb" is_pre_election: true
I20260809 04:59:20.227197 19682 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44b2ec2eccc34b4aa76ca7709bf08f3f in term 0.
I20260809 04:59:20.228410 19862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:59:20.228593 19862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:59:20.228703 19862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:20.232415 19862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } }
I20260809 04:59:20.234633 19682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0168cb199e774099a22d75f406ae68fb"
I20260809 04:59:20.234881 19682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:20.235913 19862 leader_election.cc:288] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [CANDIDATE]: Term 1 election: Requested vote from peers 0168cb199e774099a22d75f406ae68fb (127.29.243.253:46869), d410fd0584314486b989d4ded693bd2d (127.29.243.252:41445)
I20260809 04:59:20.236807 19794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d410fd0584314486b989d4ded693bd2d"
I20260809 04:59:20.237061 19794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:20.238649 19682 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44b2ec2eccc34b4aa76ca7709bf08f3f in term 1.
I20260809 04:59:20.240581 19794 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44b2ec2eccc34b4aa76ca7709bf08f3f in term 1.
I20260809 04:59:20.241034 19531 leader_election.cc:302] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0168cb199e774099a22d75f406ae68fb, 44b2ec2eccc34b4aa76ca7709bf08f3f; no voters:
I20260809 04:59:20.241629 19862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:59:20.249465 19862 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [term 1 LEADER]: Becoming Leader. State: Replica: 44b2ec2eccc34b4aa76ca7709bf08f3f, State: Running, Role: LEADER
I20260809 04:59:20.250216 19862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } }
I20260809 04:59:20.264122 19944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44b2ec2eccc34b4aa76ca7709bf08f3f. Latest consensus state: current_term: 1 leader_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } } }
I20260809 04:59:20.264804 19944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:20.270030 19952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:59:20.273120 19952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:59:20.293438 19794 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [term 1 FOLLOWER]: Refusing update from remote peer 44b2ec2eccc34b4aa76ca7709bf08f3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:59:20.294638 19944 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:59:20.305030 19682 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [term 1 FOLLOWER]: Refusing update from remote peer 44b2ec2eccc34b4aa76ca7709bf08f3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:59:20.306088 19944 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:59:20.338279 19922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44b2ec2eccc34b4aa76ca7709bf08f3f. Latest consensus state: current_term: 1 leader_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } } }
I20260809 04:59:20.338636 19922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:59:20.342850 19953 mvcc.cc:205] Tried to move back new op lower bound from 7316486390947905536 to 7316486390791012352. Current Snapshot: MvccSnapshot[applied={T|T < 7316486390947905536}]
I20260809 04:59:20.347389 19959 mvcc.cc:205] Tried to move back new op lower bound from 7316486390947905536 to 7316486390791012352. Current Snapshot: MvccSnapshot[applied={T|T < 7316486390947905536}]
I20260809 04:59:20.354298 19862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } } }
I20260809 04:59:20.354668 19862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:20.359421 19952 catalog_manager.cc:1347] Generated new cluster ID: bb02a955ea2b401983f430abffb86ed2
I20260809 04:59:20.359536 19952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:59:20.361464 19922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } } }
I20260809 04:59:20.361748 19922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:59:20.372594 19889 sys_catalog.cc:455] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44b2ec2eccc34b4aa76ca7709bf08f3f. Latest consensus state: current_term: 1 leader_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } } }
I20260809 04:59:20.372937 19889 sys_catalog.cc:458] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:59:20.380323 19952 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:59:20.382405 19952 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:59:20.399984 19968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } } }
I20260809 04:59:20.400540 19968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:20.401678 19954 mvcc.cc:205] Tried to move back new op lower bound from 7316486390947905536 to 7316486390791012352. Current Snapshot: MvccSnapshot[applied={T|T < 7316486390947905536}]
I20260809 04:59:20.402494 19952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44b2ec2eccc34b4aa76ca7709bf08f3f: Generated new TSK 0
I20260809 04:59:20.403939 19952 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:59:20.435053 19889 sys_catalog.cc:455] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b2ec2eccc34b4aa76ca7709bf08f3f" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41499 } } peers { permanent_uuid: "0168cb199e774099a22d75f406ae68fb" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 46869 } } peers { permanent_uuid: "d410fd0584314486b989d4ded693bd2d" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 41445 } } }
I20260809 04:59:20.435418 19889 sys_catalog.cc:458] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:59:21.091302 19915 catalog_manager.cc:1259] Loaded cluster ID: bb02a955ea2b401983f430abffb86ed2
I20260809 04:59:21.091444 19915 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d: loading cluster ID for follower catalog manager: success
I20260809 04:59:21.094092 19915 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d: acquiring CA information for follower catalog manager: success
I20260809 04:59:21.096061 19915 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d410fd0584314486b989d4ded693bd2d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:59:21.189248 19942 catalog_manager.cc:1259] Loaded cluster ID: bb02a955ea2b401983f430abffb86ed2
I20260809 04:59:21.189365 19942 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb: loading cluster ID for follower catalog manager: success
I20260809 04:59:21.192088 19942 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb: acquiring CA information for follower catalog manager: success
I20260809 04:59:21.194223 19942 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0168cb199e774099a22d75f406ae68fb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:59:21.543509 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 19508
I20260809 04:59:21.544639 19596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:21.656201 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 19508
I20260809 04:59:21.687285 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 19614
I20260809 04:59:21.688586 19710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:21.796175 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 19614
I20260809 04:59:21.827641 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 19732
I20260809 04:59:21.828904 19833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:21.935842 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 19732
2026-08-09T04:59:21Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5110 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:59:22 dist-test-slave-21b9 krb5kdc[19995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:59:22 dist-test-slave-21b9 krb5kdc[19995](info): set up 2 sockets
Aug 09 04:59:22 dist-test-slave-21b9 krb5kdc[19995](info): commencing operation
krb5kdc: starting...
W20260809 04:59:23.178529 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:59:23 dist-test-slave-21b9 krb5kdc[19995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:59:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:59:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:59:23.456339 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35579
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:32979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:35579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:23.794248 20016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:23.794529 20016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:23.794617 20016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:23.801508 20016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:59:23.801605 20016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:23.801656 20016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:23.801712 20016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:59:23.801759 20016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:59:23.811298 20016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:32979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:35579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:23.812521 20016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:23.813767 20016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:23.824885 20023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:23.824962 20024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:23.824983 20026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:23.825896 20016 server_base.cc:1034] running on GCE node
I20260809 04:59:23.826653 20016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:23.828025 20016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:23.829227 20016 hybrid_clock.cc:648] HybridClock initialized: now 1786251563829179 us; error 32 us; skew 500 ppm
I20260809 04:59:23.829540 20016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:23 dist-test-slave-21b9 krb5kdc[19995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251563, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:23.835186 20016 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:59:23.836683 20016 webserver.cc:460] Webserver started at http://127.29.243.254:45645/ using document root <none> and password file <none>
I20260809 04:59:23.837328 20016 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:23.837435 20016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:23.837698 20016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:23.840658 20016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "393e1251152447ea8f69fb15977f8ec9"
format_stamp: "Formatted at 2026-08-09 04:59:23 on dist-test-slave-21b9"
server_key: "dbdcb7cf26c729a902701a1d6f09ce25"
server_key_iv: "014f4b71d0a60e2802e18ae6c886e8fc"
server_key_version: "encryptionkey@0"
I20260809 04:59:23.841338 20016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "393e1251152447ea8f69fb15977f8ec9"
format_stamp: "Formatted at 2026-08-09 04:59:23 on dist-test-slave-21b9"
server_key: "dbdcb7cf26c729a902701a1d6f09ce25"
server_key_iv: "014f4b71d0a60e2802e18ae6c886e8fc"
server_key_version: "encryptionkey@0"
I20260809 04:59:23.846232 20016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:59:23.849584 20037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:23.850811 20016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 04:59:23.850962 20016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "393e1251152447ea8f69fb15977f8ec9"
format_stamp: "Formatted at 2026-08-09 04:59:23 on dist-test-slave-21b9"
server_key: "dbdcb7cf26c729a902701a1d6f09ce25"
server_key_iv: "014f4b71d0a60e2802e18ae6c886e8fc"
server_key_version: "encryptionkey@0"
I20260809 04:59:23.851127 20016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:23.872489 20016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:23.876674 20016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:23.876947 20016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:23.894193 20016 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:35579
I20260809 04:59:23.894213 20134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:35579 every 8 connection(s)
I20260809 04:59:23.895519 20016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:59:23.899231 20135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:23.903406 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 20016
I20260809 04:59:23.903643 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:59:23.904515 30671 external_mini_cluster.cc:1428] Setting key f1f69de50ced0383285a30374523e40f
I20260809 04:59:23.910894 20135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9: Bootstrap starting.
I20260809 04:59:23.914593 20135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:23.915799 20135 log.cc:824] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:23.919129 20135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9: No bootstrap required, opened a new log
Aug 09 04:59:23 dist-test-slave-21b9 krb5kdc[19995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:23.926795 20135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393e1251152447ea8f69fb15977f8ec9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35579 } }
I20260809 04:59:23.927260 20135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:23.927392 20135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393e1251152447ea8f69fb15977f8ec9, State: Initialized, Role: FOLLOWER
I20260809 04:59:23.927999 20135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393e1251152447ea8f69fb15977f8ec9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35579 } }
I20260809 04:59:23.928226 20135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:59:23.928361 20135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:59:23.928447 20138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:23.906475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.29.243.254:35579)
0809 04:59:23.907230 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:23.907244 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:23.907309 (+ 65us) server_negotiation.cc:407] Connection header received
0809 04:59:23.909343 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:23.909370 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:23.909782 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:23.910101 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:23.912006 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:23.913355 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:23.915243 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:23.915997 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:23.916138 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:23.921029 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:23.921040 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:23.921055 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:23.921093 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:23.924152 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:23.924873 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:23.924880 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:23.924886 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:23.925074 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:23.925480 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:23.925484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:23.925486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:23.926048 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:23.926329 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:23.926787 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 04:59:23.927228 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":357,"threads_started":1}
I20260809 04:59:23.928524 20135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:23.931491 20135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393e1251152447ea8f69fb15977f8ec9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35579 } }
I20260809 04:59:23.932076 20135 leader_election.cc:302] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 393e1251152447ea8f69fb15977f8ec9; no voters:
I20260809 04:59:23.932468 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:59:23.932737 20135 leader_election.cc:288] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:59:23.932845 20140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:59:23.935143 20135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:59:23.935215 20140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [term 1 LEADER]: Becoming Leader. State: Replica: 393e1251152447ea8f69fb15977f8ec9, State: Running, Role: LEADER
I20260809 04:59:23.935979 20140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393e1251152447ea8f69fb15977f8ec9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35579 } }
I20260809 04:59:23.939282 20144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "393e1251152447ea8f69fb15977f8ec9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393e1251152447ea8f69fb15977f8ec9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35579 } } }
I20260809 04:59:23.939854 20144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:23.939729 20145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 393e1251152447ea8f69fb15977f8ec9. Latest consensus state: current_term: 1 leader_uuid: "393e1251152447ea8f69fb15977f8ec9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "393e1251152447ea8f69fb15977f8ec9" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35579 } } }
I20260809 04:59:23.940003 20145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:23.940729 20161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:59:23.945935 20161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:59:23.955078 20161 catalog_manager.cc:1347] Generated new cluster ID: cedc07de352a4c9a965e91762dc0c175
I20260809 04:59:23.955257 20161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:59:23.968477 20161 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:59:23.969758 20161 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:59:23.977970 20161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 393e1251152447ea8f69fb15977f8ec9: Generated new TSK 0
I20260809 04:59:23.978744 20161 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:59:23.992607 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 20016
I20260809 04:59:23.993696 20127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:24.086823 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 20016
2026-08-09T04:59:24Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2144 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:59:24 dist-test-slave-21b9 krb5kdc[20179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:59:24 dist-test-slave-21b9 krb5kdc[20179](info): set up 2 sockets
Aug 09 04:59:24 dist-test-slave-21b9 krb5kdc[20179](info): commencing operation
krb5kdc: starting...
W20260809 04:59:25.343186 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.003s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:59:25 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:59:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:59:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:59:25.622526 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:33181
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:33181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:25.958673 20197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:25.958962 20197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:25.959051 20197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:25.966240 20197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:59:25.966359 20197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:25.966415 20197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:25.966471 20197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:59:25.966516 20197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:59:25.976429 20197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:33181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:33181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:25.977761 20197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:25.979036 20197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:25.990391 20208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:25.990401 20209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:25.990556 20211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:25.991102 20197 server_base.cc:1034] running on GCE node
I20260809 04:59:25.991907 20197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:25.993296 20197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:25.994510 20197 hybrid_clock.cc:648] HybridClock initialized: now 1786251565994444 us; error 54 us; skew 500 ppm
I20260809 04:59:25.994832 20197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:25 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251565, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:25.997967 20197 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:59:25.999539 20197 webserver.cc:460] Webserver started at http://127.29.243.254:37933/ using document root <none> and password file <none>
I20260809 04:59:26.000243 20197 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:26.000356 20197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:26.000674 20197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:26.003846 20197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "755c423f3ff545b99f6275c07b92a6eb"
format_stamp: "Formatted at 2026-08-09 04:59:25 on dist-test-slave-21b9"
server_key: "292a056e9843b226418cf3bb0d342d6e"
server_key_iv: "5a45917c04d6be601bdff82025e7185e"
server_key_version: "encryptionkey@0"
I20260809 04:59:26.004652 20197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "755c423f3ff545b99f6275c07b92a6eb"
format_stamp: "Formatted at 2026-08-09 04:59:25 on dist-test-slave-21b9"
server_key: "292a056e9843b226418cf3bb0d342d6e"
server_key_iv: "5a45917c04d6be601bdff82025e7185e"
server_key_version: "encryptionkey@0"
I20260809 04:59:26.010669 20197 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260809 04:59:26.019062 20224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:26.020712 20197 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260809 04:59:26.020884 20197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "755c423f3ff545b99f6275c07b92a6eb"
format_stamp: "Formatted at 2026-08-09 04:59:25 on dist-test-slave-21b9"
server_key: "292a056e9843b226418cf3bb0d342d6e"
server_key_iv: "5a45917c04d6be601bdff82025e7185e"
server_key_version: "encryptionkey@0"
I20260809 04:59:26.021095 20197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:26.033200 20197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:26.038743 20197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:26.039068 20197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:26.057497 20197 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:33181
I20260809 04:59:26.057538 20312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:33181 every 8 connection(s)
I20260809 04:59:26.058954 20197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:59:26.059865 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 20197
I20260809 04:59:26.060115 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:59:26.060974 30671 external_mini_cluster.cc:1428] Setting key 03002f44b269980c6ba6d991271e0744
I20260809 04:59:26.064266 20314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:59:26 dist-test-slave-21b9 krb5kdc[20179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:26.077446 20314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Bootstrap starting.
I20260809 04:59:26.080832 20314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:26.081875 20314 log.cc:824] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:26.084798 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:26.063118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.29.243.254:33181)
0809 04:59:26.063908 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:26.063920 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:26.063963 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:59:26.065572 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:26.065595 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:26.065958 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:26.066266 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:26.068107 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:26.069502 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:26.071390 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:26.072477 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:26.072610 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:26.077381 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:26.077390 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:26.077405 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:26.077448 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:26.080417 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:26.081099 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:26.081110 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:26.081117 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:26.081293 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:26.081681 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:26.081685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:26.081687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:26.082239 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:26.082573 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:26.083044 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 04:59:26.083507 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":371,"threads_started":1}
I20260809 04:59:26.085320 20314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: No bootstrap required, opened a new log
I20260809 04:59:26.093262 20314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:26.093637 20314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:26.093739 20314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755c423f3ff545b99f6275c07b92a6eb, State: Initialized, Role: FOLLOWER
I20260809 04:59:26.094393 20314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:26.094619 20314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:59:26.094713 20314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:59:26.094832 20314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:26.096943 20314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:26.097402 20314 leader_election.cc:302] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 755c423f3ff545b99f6275c07b92a6eb; no voters:
I20260809 04:59:26.097952 20314 leader_election.cc:288] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:59:26.098104 20323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:59:26.099013 20323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 1 LEADER]: Becoming Leader. State: Replica: 755c423f3ff545b99f6275c07b92a6eb, State: Running, Role: LEADER
I20260809 04:59:26.099560 20323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:26.099888 20314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:59:26.102053 20325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 755c423f3ff545b99f6275c07b92a6eb. Latest consensus state: current_term: 1 leader_uuid: "755c423f3ff545b99f6275c07b92a6eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } } }
I20260809 04:59:26.102356 20325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:26.102833 20324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "755c423f3ff545b99f6275c07b92a6eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } } }
I20260809 04:59:26.103116 20324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:26.103320 20333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:59:26.114459 20333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:59:26.122637 20333 catalog_manager.cc:1347] Generated new cluster ID: 788ba0c50b4040b8937664e988a979ed
I20260809 04:59:26.122764 20333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:59:26.163226 20333 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:59:26.165271 20333 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:59:26.178658 20333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Generated new TSK 0
I20260809 04:59:26.179587 20333 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:59:26.254879 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:33181
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:26.606513 20359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:26.606817 20359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:26.606973 20359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:26.614526 20359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:26.614650 20359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:26.614799 20359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:59:26.624964 20359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:26.626436 20359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:26.627826 20359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:26.641348 20368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:26.641309 20370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:26.641275 20372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:26.641917 20359 server_base.cc:1034] running on GCE node
I20260809 04:59:26.642584 20359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:26.643621 20359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:26.644843 20359 hybrid_clock.cc:648] HybridClock initialized: now 1786251566644786 us; error 47 us; skew 500 ppm
I20260809 04:59:26.645175 20359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:26 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251566, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:26.648304 20359 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:59:26.649744 20359 webserver.cc:460] Webserver started at http://127.29.243.193:42127/ using document root <none> and password file <none>
I20260809 04:59:26.650440 20359 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:26.650563 20359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:26.650859 20359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:26.653836 20359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "0fcf4105d19248f8be2da5daae36bff0"
format_stamp: "Formatted at 2026-08-09 04:59:26 on dist-test-slave-21b9"
server_key: "1af7a6671a8cdb83171f3ff2399ded85"
server_key_iv: "8ed40cd377d35ed1b3b03dfb63aa7b4f"
server_key_version: "encryptionkey@0"
I20260809 04:59:26.654626 20359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "0fcf4105d19248f8be2da5daae36bff0"
format_stamp: "Formatted at 2026-08-09 04:59:26 on dist-test-slave-21b9"
server_key: "1af7a6671a8cdb83171f3ff2399ded85"
server_key_iv: "8ed40cd377d35ed1b3b03dfb63aa7b4f"
server_key_version: "encryptionkey@0"
I20260809 04:59:26.659798 20359 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 04:59:26.663352 20380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:26.664739 20359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:59:26.664901 20359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "0fcf4105d19248f8be2da5daae36bff0"
format_stamp: "Formatted at 2026-08-09 04:59:26 on dist-test-slave-21b9"
server_key: "1af7a6671a8cdb83171f3ff2399ded85"
server_key_iv: "8ed40cd377d35ed1b3b03dfb63aa7b4f"
server_key_version: "encryptionkey@0"
I20260809 04:59:26.665094 20359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:26.680305 20359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:26.684129 20359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:26.684401 20359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:26.685360 20359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:26.686721 20359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:26.686801 20359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:26.686935 20359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:26.686997 20359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:26.718084 20359 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:43573
I20260809 04:59:26.718130 20569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:43573 every 8 connection(s)
I20260809 04:59:26.719406 20359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:59:26.724587 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 20359
I20260809 04:59:26.724802 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:59:26.725489 30671 external_mini_cluster.cc:1428] Setting key 30dd8c4d30a6f1a93d3515d813b7c7af
Aug 09 04:59:26 dist-test-slave-21b9 krb5kdc[20179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251566, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:26.739130 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:26.722078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:44293 (local address 127.29.243.254:33181)
0809 04:59:26.722552 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:26.722557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:26.723617 (+ 1060us) server_negotiation.cc:407] Connection header received
0809 04:59:26.725655 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:26.725661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:26.725741 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:26.725848 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:26.727649 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:26.728391 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:26.729940 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:26.730329 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:26.730442 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:26.733742 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:26.733747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:26.733750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:26.733782 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:26.736190 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:26.736889 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:26.736893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:26.736895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:26.737012 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:26.737429 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:26.737433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:26.737435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:26.737693 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:26.737827 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:26.738550 (+ 723us) server_negotiation.cc:299] Negotiation successful
0809 04:59:26.738772 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":296,"threads_started":1}
I20260809 04:59:26.740139 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:26.722627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:44293)
0809 04:59:26.723455 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:26.723497 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:26.725357 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:26.726108 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:26.726118 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:26.726713 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:26.727487 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:26.727508 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:26.728554 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:26.728558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:26.729792 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:26.729804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:26.730554 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:26.730561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:26.730783 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:26.731419 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:26.731443 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:26.733537 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:26.736333 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:26.736343 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:26.736360 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:26.736766 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:26.737113 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:26.737119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:26.737123 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:26.737314 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:26.737817 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:26.737825 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:26.738164 (+ 339us) client_negotiation.cc:769] Sending connection context
0809 04:59:26.738406 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 04:59:26.738958 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":281,"threads_started":1}
I20260809 04:59:26.741649 20570 heartbeater.cc:347] Connected to a master server at 127.29.243.254:33181
I20260809 04:59:26.741947 20570 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:26.742689 20570 heartbeater.cc:511] Master 127.29.243.254:33181 requested a full tablet report, sending...
I20260809 04:59:26.744745 20253 ts_manager.cc:194] Registered new tserver with Master: 0fcf4105d19248f8be2da5daae36bff0 (127.29.243.193:43573)
I20260809 04:59:26.746186 20253 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:44293
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:59:26.785867 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:33181
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:27.135946 20578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:27.136215 20578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:27.136310 20578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:27.143777 20578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:27.143884 20578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:27.144016 20578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:59:27.153666 20578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:27.154939 20578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:27.156417 20578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:27.169472 20589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:27.170317 20578 server_base.cc:1034] running on GCE node
W20260809 04:59:27.175258 20592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:27.175637 20590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:27.176431 20578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:27.177455 20578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:27.178658 20578 hybrid_clock.cc:648] HybridClock initialized: now 1786251567178606 us; error 46 us; skew 500 ppm
I20260809 04:59:27.178985 20578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:27 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251567, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:27.182120 20578 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:59:27.183696 20578 webserver.cc:460] Webserver started at http://127.29.243.194:39363/ using document root <none> and password file <none>
I20260809 04:59:27.184394 20578 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:27.184518 20578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:27.184818 20578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:27.187903 20578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60"
format_stamp: "Formatted at 2026-08-09 04:59:27 on dist-test-slave-21b9"
server_key: "ff99fd512104060da4c7020f0ed36c64"
server_key_iv: "3d81b99f94ab3c67bc428f4491dcfc53"
server_key_version: "encryptionkey@0"
I20260809 04:59:27.188678 20578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60"
format_stamp: "Formatted at 2026-08-09 04:59:27 on dist-test-slave-21b9"
server_key: "ff99fd512104060da4c7020f0ed36c64"
server_key_iv: "3d81b99f94ab3c67bc428f4491dcfc53"
server_key_version: "encryptionkey@0"
I20260809 04:59:27.193883 20578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:59:27.197453 20599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:27.198812 20578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 04:59:27.198959 20578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60"
format_stamp: "Formatted at 2026-08-09 04:59:27 on dist-test-slave-21b9"
server_key: "ff99fd512104060da4c7020f0ed36c64"
server_key_iv: "3d81b99f94ab3c67bc428f4491dcfc53"
server_key_version: "encryptionkey@0"
I20260809 04:59:27.199144 20578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:27.216902 20578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:27.221091 20578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:27.221375 20578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:27.222378 20578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:27.223867 20578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:27.223941 20578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:27.224069 20578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:27.224128 20578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:27.257616 20578 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:33517
I20260809 04:59:27.257634 20771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:33517 every 8 connection(s)
I20260809 04:59:27.258906 20578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:59:27.267983 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 20578
I20260809 04:59:27.268213 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:59:27.268680 30671 external_mini_cluster.cc:1428] Setting key d5b3d77b0b2e2c278eed282524f9464e
Aug 09 04:59:27 dist-test-slave-21b9 krb5kdc[20179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251567, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:27.278720 20776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.261449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:36577 (local address 127.29.243.254:33181)
0809 04:59:27.261923 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:27.261928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:27.262901 (+ 973us) server_negotiation.cc:407] Connection header received
0809 04:59:27.264774 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:27.264780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:27.264860 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:27.264960 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:27.266486 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.267198 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.268986 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.269499 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.269624 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.273150 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:27.273155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:27.273158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:27.273190 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:27.275691 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:27.276402 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:27.276407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:27.276410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:27.276557 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:27.276975 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:27.276978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:27.276980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:27.277244 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:27.277382 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:27.278090 (+ 708us) server_negotiation.cc:299] Negotiation successful
0809 04:59:27.278329 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":222,"threads_started":1}
I20260809 04:59:27.279734 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.262007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:36577)
0809 04:59:27.262725 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:27.262773 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:27.264496 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:27.265217 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:27.265226 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:27.265734 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:27.266314 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.266332 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.267376 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.267380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.268844 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.268858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.269740 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.269749 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.269981 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.270694 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:27.270722 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:27.272915 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:27.275824 (+ 2909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:27.275834 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:27.275853 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:27.276278 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:27.276668 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:27.276673 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:27.276676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:27.276865 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:27.277376 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:27.277384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:27.277727 (+ 343us) client_negotiation.cc:769] Sending connection context
0809 04:59:27.277979 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 04:59:27.278528 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":271,"threads_started":1}
I20260809 04:59:27.281184 20772 heartbeater.cc:347] Connected to a master server at 127.29.243.254:33181
I20260809 04:59:27.281522 20772 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:27.282279 20772 heartbeater.cc:511] Master 127.29.243.254:33181 requested a full tablet report, sending...
I20260809 04:59:27.283777 20253 ts_manager.cc:194] Registered new tserver with Master: 04c9fc9b2d96493b82b0dbd7c22c2d60 (127.29.243.194:33517)
I20260809 04:59:27.284576 20253 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:36577
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:59:27.329991 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:33181
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:27.670527 20782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:27.670842 20782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:27.670979 20782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:27.678236 20782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:27.678368 20782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:27.678527 20782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:59:27.688148 20782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:27.689530 20782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:27.690927 20782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:27.703954 20787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:27.703994 20793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:27.704041 20791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:27.705084 20782 server_base.cc:1034] running on GCE node
I20260809 04:59:27.705687 20782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:27.706678 20782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:27.707898 20782 hybrid_clock.cc:648] HybridClock initialized: now 1786251567707825 us; error 48 us; skew 500 ppm
I20260809 04:59:27.708241 20782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:27 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251567, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:27.711365 20782 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:59:27.712813 20782 webserver.cc:460] Webserver started at http://127.29.243.195:35151/ using document root <none> and password file <none>
I20260809 04:59:27.713496 20782 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:27.713621 20782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:27.713922 20782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:27.716917 20782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "bb93c6824e914466bdc4ebdfbb6782bb"
format_stamp: "Formatted at 2026-08-09 04:59:27 on dist-test-slave-21b9"
server_key: "a100cb2fad3c4e76b18a8fa0d032c779"
server_key_iv: "cf0dc8a4d675de62375312f1c80b099f"
server_key_version: "encryptionkey@0"
I20260809 04:59:27.717703 20782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "bb93c6824e914466bdc4ebdfbb6782bb"
format_stamp: "Formatted at 2026-08-09 04:59:27 on dist-test-slave-21b9"
server_key: "a100cb2fad3c4e76b18a8fa0d032c779"
server_key_iv: "cf0dc8a4d675de62375312f1c80b099f"
server_key_version: "encryptionkey@0"
I20260809 04:59:27.722932 20782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 04:59:27.726599 20804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:27.728003 20782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 04:59:27.728161 20782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "bb93c6824e914466bdc4ebdfbb6782bb"
format_stamp: "Formatted at 2026-08-09 04:59:27 on dist-test-slave-21b9"
server_key: "a100cb2fad3c4e76b18a8fa0d032c779"
server_key_iv: "cf0dc8a4d675de62375312f1c80b099f"
server_key_version: "encryptionkey@0"
I20260809 04:59:27.728345 20782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:27.739373 20782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:27.743402 20782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:27.743660 20782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:27.744545 20782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:27.745839 20782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:27.745926 20782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:27.746042 20782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:27.746096 20782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:27.749044 20570 heartbeater.cc:503] Master 127.29.243.254:33181 was elected leader, sending a full tablet report...
I20260809 04:59:27.776516 20782 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:45017
I20260809 04:59:27.776556 20977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:45017 every 8 connection(s)
I20260809 04:59:27.777796 20782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:59:27.787695 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 20782
I20260809 04:59:27.787878 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:59:27.788316 30671 external_mini_cluster.cc:1428] Setting key 8b2ae1058716645c9ba0a58afa18ed53
Aug 09 04:59:27 dist-test-slave-21b9 krb5kdc[20179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251567, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:27.798076 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.780406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:44611 (local address 127.29.243.254:33181)
0809 04:59:27.781018 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:27.781022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:27.781913 (+ 891us) server_negotiation.cc:407] Connection header received
0809 04:59:27.783784 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:27.783790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:27.783874 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:27.783982 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:27.785530 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.786302 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.787942 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.788345 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.788478 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.792749 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:27.792754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:27.792758 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:27.792793 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:27.795319 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:27.796040 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:27.796044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:27.796046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:27.796163 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:27.796531 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:27.796534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:27.796536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:27.796794 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:27.796914 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:27.797603 (+ 689us) server_negotiation.cc:299] Negotiation successful
0809 04:59:27.797781 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":300,"threads_started":1}
I20260809 04:59:27.799038 20981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.780981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:44611)
0809 04:59:27.781739 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:27.781787 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:27.783515 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:27.784226 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:27.784235 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:27.784778 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:27.785354 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.785373 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.786466 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.786470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.787770 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.787787 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.788633 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.788642 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.788940 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.789803 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:27.789838 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:27.792533 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:27.795455 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:27.795463 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:27.795478 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:27.795930 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:27.796243 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:27.796246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:27.796248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:27.796447 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:27.796888 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:27.796894 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:27.797226 (+ 332us) client_negotiation.cc:769] Sending connection context
0809 04:59:27.797455 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 04:59:27.797863 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":266,"threads_started":1}
I20260809 04:59:27.800485 20978 heartbeater.cc:347] Connected to a master server at 127.29.243.254:33181
I20260809 04:59:27.800800 20978 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:27.801528 20978 heartbeater.cc:511] Master 127.29.243.254:33181 requested a full tablet report, sending...
I20260809 04:59:27.803026 20253 ts_manager.cc:194] Registered new tserver with Master: bb93c6824e914466bdc4ebdfbb6782bb (127.29.243.195:45017)
I20260809 04:59:27.804085 20253 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:44611
I20260809 04:59:27.804585 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:59:27.821549 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.809187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56564 (local address 127.29.243.254:33181)
0809 04:59:27.809346 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:27.809350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:27.809781 (+ 431us) server_negotiation.cc:407] Connection header received
0809 04:59:27.809937 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:27.809941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:27.810000 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:27.810075 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:27.811548 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.812243 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.813493 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.813874 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.813975 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.816236 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:27.816241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:27.816244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:27.816274 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:27.819102 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:27.819826 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:27.819830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:27.819833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:27.819955 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:27.820246 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:27.820249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:27.820251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:27.820554 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:27.820751 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:27.821079 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 04:59:27.821247 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:59:27.825768 20253 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:59:27.832772 20253 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:59:27.856434 20993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.845060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43573 (local address 127.0.0.1:53900)
0809 04:59:27.846685 (+ 1625us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:27.846731 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:27.846930 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:27.848094 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:27.848101 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:27.848133 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:27.848588 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.848604 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.850807 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.850814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.853346 (+ 2532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.853364 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.855450 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.855458 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.855663 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.855727 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 04:59:27.855844 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 04:59:27.856031 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1341,"thread_start_us":1166,"threads_started":1}
I20260809 04:59:27.857721 20994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.845200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53900 (local address 127.29.243.193:43573)
0809 04:59:27.846261 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:27.846272 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:27.847131 (+ 859us) server_negotiation.cc:407] Connection header received
0809 04:59:27.847254 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:27.847263 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:27.847587 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:27.847783 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:27.848836 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.850618 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.853564 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.855181 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.856936 (+ 1755us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.857131 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:27.857211 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 04:59:27.857392 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":885,"thread_start_us":709,"threads_started":1}
I20260809 04:59:27.862298 20461 tablet_service.cc:1467] Processing CreateTablet for tablet 1c06b1f6352f4befb6bcd39944d7ee87 (DEFAULT_TABLE table=test-table [id=a6e92b79d510444baba25c3fc426922f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:27.863662 20461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c06b1f6352f4befb6bcd39944d7ee87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:27.865913 20997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.845062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:45017 (local address 127.0.0.1:50958)
0809 04:59:27.857186 (+ 12124us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:27.857208 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:27.857338 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:27.858919 (+ 1581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:27.858924 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:27.858949 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:27.859347 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.859362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.861050 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.861055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.863219 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.863236 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.865062 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.865069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.865234 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.865269 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 04:59:27.865361 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:59:27.865488 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11970,"mutex_wait_us":824,"thread_start_us":11337,"threads_started":1}
I20260809 04:59:27.865950 20995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.845586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.29.243.195:45017)
0809 04:59:27.858258 (+ 12672us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:27.858267 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:27.858306 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:59:27.858412 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:27.858420 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:27.858701 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:27.858902 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:27.859570 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.860887 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.863437 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.864849 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.865042 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.865213 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:27.865447 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 04:59:27.865622 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12486,"thread_start_us":12301,"threads_started":1}
I20260809 04:59:27.866798 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.847303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46036 (local address 127.29.243.194:33517)
0809 04:59:27.858587 (+ 11284us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:27.858593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:27.858616 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:59:27.858698 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:27.858703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:27.858870 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:27.859073 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:27.859937 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.861438 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.863857 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.865276 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.865429 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.865585 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:27.866297 (+ 712us) server_negotiation.cc:299] Negotiation successful
0809 04:59:27.866468 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11150,"thread_start_us":11011,"threads_started":1}
I20260809 04:59:27.867484 20996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.847018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:33517 (local address 127.0.0.1:46036)
0809 04:59:27.857526 (+ 10508us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:27.857543 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:27.857654 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:27.859329 (+ 1675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:27.859333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:27.859360 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:27.859748 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.859761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.861587 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.861592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.863681 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.863694 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.865755 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.865762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.865934 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.865966 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:59:27.866062 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 04:59:27.866188 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10380,"thread_start_us":9839,"threads_started":1}
I20260809 04:59:27.870398 20669 tablet_service.cc:1467] Processing CreateTablet for tablet 1c06b1f6352f4befb6bcd39944d7ee87 (DEFAULT_TABLE table=test-table [id=a6e92b79d510444baba25c3fc426922f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:27.870206 20871 tablet_service.cc:1467] Processing CreateTablet for tablet 1c06b1f6352f4befb6bcd39944d7ee87 (DEFAULT_TABLE table=test-table [id=a6e92b79d510444baba25c3fc426922f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:27.871582 20669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c06b1f6352f4befb6bcd39944d7ee87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:27.871688 20871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c06b1f6352f4befb6bcd39944d7ee87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:27.878403 21001 tablet_bootstrap.cc:492] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: Bootstrap starting.
I20260809 04:59:27.882249 21001 tablet_bootstrap.cc:654] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:27.883713 21001 log.cc:824] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:27.884605 21002 tablet_bootstrap.cc:492] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: Bootstrap starting.
I20260809 04:59:27.886094 21003 tablet_bootstrap.cc:492] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: Bootstrap starting.
I20260809 04:59:27.888067 21002 tablet_bootstrap.cc:654] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:27.889094 21002 log.cc:824] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:27.889168 21001 tablet_bootstrap.cc:492] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: No bootstrap required, opened a new log
I20260809 04:59:27.889459 21001 ts_tablet_manager.cc:1397] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260809 04:59:27.890094 21003 tablet_bootstrap.cc:654] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:27.891346 21003 log.cc:824] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:27.891650 21002 tablet_bootstrap.cc:492] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: No bootstrap required, opened a new log
I20260809 04:59:27.891893 21002 ts_tablet_manager.cc:1397] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 04:59:27.894387 21003 tablet_bootstrap.cc:492] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: No bootstrap required, opened a new log
I20260809 04:59:27.894649 21003 ts_tablet_manager.cc:1397] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 04:59:27.899638 21001 raft_consensus.cc:348] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } }
I20260809 04:59:27.900087 21001 raft_consensus.cc:374] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:27.899981 21002 raft_consensus.cc:348] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } }
I20260809 04:59:27.900235 21001 raft_consensus.cc:729] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fcf4105d19248f8be2da5daae36bff0, State: Initialized, Role: FOLLOWER
I20260809 04:59:27.900372 21002 raft_consensus.cc:374] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:27.900480 21002 raft_consensus.cc:729] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb93c6824e914466bdc4ebdfbb6782bb, State: Initialized, Role: FOLLOWER
I20260809 04:59:27.901053 21002 consensus_queue.cc:260] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } }
I20260809 04:59:27.901015 21001 consensus_queue.cc:260] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } }
I20260809 04:59:27.902925 21002 ts_tablet_manager.cc:1428] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260809 04:59:27.903286 21001 ts_tablet_manager.cc:1428] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: Time spent starting tablet: real 0.014s user 0.008s sys 0.006s
I20260809 04:59:27.903473 20978 heartbeater.cc:503] Master 127.29.243.254:33181 was elected leader, sending a full tablet report...
I20260809 04:59:27.904690 21003 raft_consensus.cc:348] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } }
I20260809 04:59:27.905136 21003 raft_consensus.cc:374] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:27.905256 21003 raft_consensus.cc:729] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c9fc9b2d96493b82b0dbd7c22c2d60, State: Initialized, Role: FOLLOWER
I20260809 04:59:27.905931 21003 consensus_queue.cc:260] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } }
I20260809 04:59:27.907886 20772 heartbeater.cc:503] Master 127.29.243.254:33181 was elected leader, sending a full tablet report...
I20260809 04:59:27.908093 21003 ts_tablet_manager.cc:1428] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
W20260809 04:59:27.971879 20571 tablet.cc:2369] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:59:27.985812 21010 raft_consensus.cc:482] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:59:27.986145 21010 raft_consensus.cc:504] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } }
I20260809 04:59:27.988018 21010 leader_election.cc:288] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fcf4105d19248f8be2da5daae36bff0 (127.29.243.193:43573), 04c9fc9b2d96493b82b0dbd7c22c2d60 (127.29.243.194:33517)
I20260809 04:59:27.993906 20994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.988580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:41735 (local address 127.29.243.193:43573)
0809 04:59:27.988805 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:27.988811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:27.989247 (+ 436us) server_negotiation.cc:407] Connection header received
0809 04:59:27.989324 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:27.989327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:27.989391 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:27.989468 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:27.989922 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.990642 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.992275 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.993053 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.993200 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.993279 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:27.993533 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 04:59:27.993658 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:59:27.993912 20981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.988580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43573 (local address 127.29.243.195:41735)
0809 04:59:27.988931 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:27.988951 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:27.989124 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:27.989513 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:27.989518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:27.989545 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:27.989766 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.989775 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.990774 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.990778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.992128 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.992140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.993200 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.993206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.993388 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.993412 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:59:27.993490 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 04:59:27.993621 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"spinlock_wait_cycles":25344}
I20260809 04:59:27.994436 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.988631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:56685 (local address 127.29.243.194:33517)
0809 04:59:27.988858 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:27.988865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:27.989569 (+ 704us) server_negotiation.cc:407] Connection header received
0809 04:59:27.989733 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:27.989737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:27.989803 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:27.989879 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:27.990408 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.991133 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.992796 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.993559 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.993696 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.993766 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:27.994098 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 04:59:27.994221 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 04:59:27.995057 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:27.988631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:33517 (local address 127.29.243.195:56685)
0809 04:59:27.989452 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:27.989466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:27.989601 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:27.989908 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:27.989912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:27.989935 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:27.990241 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.990249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.991309 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.991315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.992635 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:27.992647 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:27.993753 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:27.993759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:27.993906 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:27.993928 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:59:27.994622 (+ 694us) client_negotiation.cc:240] Negotiation successful
0809 04:59:27.994744 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":472,"threads_started":1}
I20260809 04:59:27.995046 20494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c06b1f6352f4befb6bcd39944d7ee87" candidate_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fcf4105d19248f8be2da5daae36bff0" is_pre_election: true
I20260809 04:59:27.995723 20494 raft_consensus.cc:2454] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb93c6824e914466bdc4ebdfbb6782bb in term 0.
I20260809 04:59:27.996161 20703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c06b1f6352f4befb6bcd39944d7ee87" candidate_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" is_pre_election: true
I20260809 04:59:27.996515 20809 leader_election.cc:302] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fcf4105d19248f8be2da5daae36bff0, bb93c6824e914466bdc4ebdfbb6782bb; no voters:
I20260809 04:59:27.996668 20703 raft_consensus.cc:2454] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb93c6824e914466bdc4ebdfbb6782bb in term 0.
I20260809 04:59:27.996899 21010 raft_consensus.cc:2781] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:59:27.997035 21010 raft_consensus.cc:482] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:59:27.997140 21010 raft_consensus.cc:3037] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:27.999502 21010 raft_consensus.cc:504] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } }
I20260809 04:59:28.000111 21010 leader_election.cc:288] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [CANDIDATE]: Term 1 election: Requested vote from peers 0fcf4105d19248f8be2da5daae36bff0 (127.29.243.193:43573), 04c9fc9b2d96493b82b0dbd7c22c2d60 (127.29.243.194:33517)
I20260809 04:59:28.000563 20494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c06b1f6352f4befb6bcd39944d7ee87" candidate_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fcf4105d19248f8be2da5daae36bff0"
I20260809 04:59:28.000631 20703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c06b1f6352f4befb6bcd39944d7ee87" candidate_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60"
I20260809 04:59:28.000754 20494 raft_consensus.cc:3037] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:28.000864 20703 raft_consensus.cc:3037] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:28.003178 20494 raft_consensus.cc:2454] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb93c6824e914466bdc4ebdfbb6782bb in term 1.
I20260809 04:59:28.003224 20703 raft_consensus.cc:2454] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb93c6824e914466bdc4ebdfbb6782bb in term 1.
I20260809 04:59:28.003607 20809 leader_election.cc:302] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fcf4105d19248f8be2da5daae36bff0, bb93c6824e914466bdc4ebdfbb6782bb; no voters:
I20260809 04:59:28.003903 21010 raft_consensus.cc:2781] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:59:28.004339 21010 raft_consensus.cc:686] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 1 LEADER]: Becoming Leader. State: Replica: bb93c6824e914466bdc4ebdfbb6782bb, State: Running, Role: LEADER
I20260809 04:59:28.004834 21010 consensus_queue.cc:237] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } }
I20260809 04:59:28.010144 20251 catalog_manager.cc:5495] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb reported cstate change: term changed from 0 to 1, leader changed from <none> to bb93c6824e914466bdc4ebdfbb6782bb (127.29.243.195). New cstate: current_term: 1 leader_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } health_report { overall_health: UNKNOWN } } }
W20260809 04:59:28.010951 20773 tablet.cc:2369] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:59:28.029627 20979 tablet.cc:2369] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:59:28.047178 20995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:28.041546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50972 (local address 127.29.243.195:45017)
0809 04:59:28.041759 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:28.041766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:28.041806 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:59:28.042051 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:28.042055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:28.042136 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:28.042271 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:59:28.042675 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:28.043470 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:28.045031 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:28.045421 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:28.045531 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:28.046068 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:59:28.046582 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:59:28.046727 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:28.046824 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 04:59:28.046909 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:59:28.055723 20703 raft_consensus.cc:1261] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 1 FOLLOWER]: Refusing update from remote peer bb93c6824e914466bdc4ebdfbb6782bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:59:28.055719 20494 raft_consensus.cc:1261] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0 [term 1 FOLLOWER]: Refusing update from remote peer bb93c6824e914466bdc4ebdfbb6782bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:59:28.056743 21017 consensus_queue.cc:1035] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:59:28.057680 21010 consensus_queue.cc:1035] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:59:28.069448 21023 mvcc.cc:205] Tried to move back new op lower bound from 7316486422745284608 to 7316486422552055808. Current Snapshot: MvccSnapshot[applied={T|T < 7316486422745284608}]
I20260809 04:59:28.094650 20249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:59:28.095036 20249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:59:28.099258 20249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Sending DeleteTablet for 3 replicas of tablet 1c06b1f6352f4befb6bcd39944d7ee87
I20260809 04:59:28.100518 20871 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c06b1f6352f4befb6bcd39944d7ee87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:28 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:50958
I20260809 04:59:28.100768 20669 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c06b1f6352f4befb6bcd39944d7ee87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:28 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:46036
I20260809 04:59:28.100978 20461 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c06b1f6352f4befb6bcd39944d7ee87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:28 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:53900
I20260809 04:59:28.101882 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 20359
I20260809 04:59:28.111629 21039 tablet_replica.cc:330] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: stopping tablet replica
I20260809 04:59:28.112332 21039 raft_consensus.cc:2229] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:59:28.113298 21039 raft_consensus.cc:2258] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:28.114257 21041 tablet_replica.cc:330] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: stopping tablet replica
I20260809 04:59:28.115253 21042 tablet_replica.cc:330] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: stopping tablet replica
I20260809 04:59:28.115712 21042 raft_consensus.cc:2229] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:59:28.116231 21042 raft_consensus.cc:2258] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:28.116559 21039 ts_tablet_manager.cc:1905] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:28.116920 21041 raft_consensus.cc:2229] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:59:28.117427 21041 raft_consensus.cc:2258] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:28.119818 21042 ts_tablet_manager.cc:1905] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:28.122373 21041 ts_tablet_manager.cc:1905] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:28.124342 21039 ts_tablet_manager.cc:1918] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:59:28.124575 21039 log.cc:1196] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/1c06b1f6352f4befb6bcd39944d7ee87
I20260809 04:59:28.125361 21039 ts_tablet_manager.cc:1939] T 1c06b1f6352f4befb6bcd39944d7ee87 P bb93c6824e914466bdc4ebdfbb6782bb: Deleting consensus metadata
I20260809 04:59:28.126989 21042 ts_tablet_manager.cc:1918] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:59:28.127156 21042 log.cc:1196] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/1c06b1f6352f4befb6bcd39944d7ee87
I20260809 04:59:28.127524 20230 catalog_manager.cc:4841] TS bb93c6824e914466bdc4ebdfbb6782bb (127.29.243.195:45017): tablet 1c06b1f6352f4befb6bcd39944d7ee87 (table test-table [id=a6e92b79d510444baba25c3fc426922f]) successfully deleted
I20260809 04:59:28.127737 21042 ts_tablet_manager.cc:1939] T 1c06b1f6352f4befb6bcd39944d7ee87 P 0fcf4105d19248f8be2da5daae36bff0: Deleting consensus metadata
I20260809 04:59:28.129274 20227 catalog_manager.cc:4841] TS 0fcf4105d19248f8be2da5daae36bff0 (127.29.243.193:43573): tablet 1c06b1f6352f4befb6bcd39944d7ee87 (table test-table [id=a6e92b79d510444baba25c3fc426922f]) successfully deleted
Aug 09 04:59:28 dist-test-slave-21b9 krb5kdc[20179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:59:28.131963 21041 ts_tablet_manager.cc:1918] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:59:28.132153 21041 log.cc:1196] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/1c06b1f6352f4befb6bcd39944d7ee87
I20260809 04:59:28.132759 21041 ts_tablet_manager.cc:1939] T 1c06b1f6352f4befb6bcd39944d7ee87 P 04c9fc9b2d96493b82b0dbd7c22c2d60: Deleting consensus metadata
I20260809 04:59:28.134675 20226 catalog_manager.cc:4841] TS 04c9fc9b2d96493b82b0dbd7c22c2d60 (127.29.243.194:33517): tablet 1c06b1f6352f4befb6bcd39944d7ee87 (table test-table [id=a6e92b79d510444baba25c3fc426922f]) successfully deleted
I20260809 04:59:28.139010 20994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:28.103327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53902 (local address 127.29.243.193:43573)
0809 04:59:28.103533 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:28.103541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:28.114593 (+ 11052us) server_negotiation.cc:407] Connection header received
0809 04:59:28.114839 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:28.114845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:28.114966 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:28.115100 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:28.117274 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:28.118435 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:28.125086 (+ 6651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:28.125893 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:28.126019 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:28.132951 (+ 6932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:28.132964 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:28.132980 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:28.133026 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:28.135996 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:28.136822 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:28.136833 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:28.136836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:28.136979 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:28.137392 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:28.137396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:28.137399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:28.137780 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:28.138076 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:28.138428 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 04:59:28.138663 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:59:28.139816 20559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:28.229020 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 20359
I20260809 04:59:28.263355 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 20578
Aug 09 04:59:28 dist-test-slave-21b9 krb5kdc[20179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:59:28.282773 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:28.264302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46048 (local address 127.29.243.194:33517)
0809 04:59:28.264469 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:28.264473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:28.264538 (+ 65us) server_negotiation.cc:407] Connection header received
0809 04:59:28.264737 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:28.264743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:28.264824 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:28.264933 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:28.266667 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:28.267881 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:28.269710 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:28.270346 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:28.270460 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:28.276133 (+ 5673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:28.276149 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:28.276169 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:28.276222 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:28.279978 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:28.280651 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:28.280662 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:28.280666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:28.280815 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:28.281173 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:28.281178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:28.281181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:28.281684 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:28.281989 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:28.282180 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 04:59:28.282412 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:59:28.283533 20765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:28.373356 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 20578
I20260809 04:59:28.402537 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 20782
Aug 09 04:59:28 dist-test-slave-21b9 krb5kdc[20179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:59:28.419101 20995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:28.403371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.29.243.195:45017)
0809 04:59:28.403585 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:28.403590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:28.403611 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:59:28.403755 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:28.403761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:28.403832 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:28.403926 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:28.405320 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:28.406279 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:28.407829 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:28.408280 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:28.408395 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:28.412992 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:28.413006 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:28.413025 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:28.413075 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:28.416132 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:28.416815 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:28.416823 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:28.416825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:28.416986 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:28.417396 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:28.417400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:28.417402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:28.417816 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:28.418121 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:28.418470 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 04:59:28.418715 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:59:28.419946 20973 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:28.542613 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 20782
I20260809 04:59:28.571352 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 20197
I20260809 04:59:28.572654 20306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:28.682940 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 20197
I20260809 04:59:28.715219 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:33181
--webserver_interface=127.29.243.254
--webserver_port=37933
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:33181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:29.059432 21053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:29.059755 21053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:29.059855 21053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:29.067142 21053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:59:29.067255 21053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:29.067346 21053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:29.067409 21053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:59:29.067454 21053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:59:29.077483 21053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:33181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:33181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=37933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:29.078794 21053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:29.080118 21053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:29.106268 21064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:29.106213 21068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:29.106228 21065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:29.112493 21053 server_base.cc:1034] running on GCE node
I20260809 04:59:29.113438 21053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:29.115408 21053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:29.116626 21053 hybrid_clock.cc:648] HybridClock initialized: now 1786251569116567 us; error 45 us; skew 500 ppm
I20260809 04:59:29.117010 21053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:29 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251569, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:29.125099 21053 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:59:29.127099 21053 webserver.cc:460] Webserver started at http://127.29.243.254:37933/ using document root <none> and password file <none>
I20260809 04:59:29.128131 21053 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:29.128298 21053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:29.134895 21053 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:59:29.138048 21083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:29.139684 21053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:59:29.139849 21053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "755c423f3ff545b99f6275c07b92a6eb"
format_stamp: "Formatted at 2026-08-09 04:59:25 on dist-test-slave-21b9"
server_key: "292a056e9843b226418cf3bb0d342d6e"
server_key_iv: "5a45917c04d6be601bdff82025e7185e"
server_key_version: "encryptionkey@0"
I20260809 04:59:29.140944 21053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:29.162472 21053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:29.166885 21053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:29.167135 21053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:29.186210 21053 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:33181
I20260809 04:59:29.186220 21160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:33181 every 8 connection(s)
I20260809 04:59:29.187846 21053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:59:29.193751 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 21053
I20260809 04:59:29.194581 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:43573
--local_ip_for_outbound_sockets=127.29.243.193
--tserver_master_addrs=127.29.243.254:33181
--webserver_port=42127
--webserver_interface=127.29.243.193
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:59:29.195387 21161 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:59:29.198833 21161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Bootstrap starting.
I20260809 04:59:29.218986 21161 log.cc:824] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:29.231757 21161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:59:29.232302 21161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Bootstrap complete.
I20260809 04:59:29.242024 21161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:29.243249 21161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755c423f3ff545b99f6275c07b92a6eb, State: Initialized, Role: FOLLOWER
I20260809 04:59:29.244120 21161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:29.244407 21161 raft_consensus.cc:388] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:59:29.244534 21161 raft_consensus.cc:482] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:59:29.244704 21161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:59:29.248857 21161 raft_consensus.cc:504] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:29.249436 21161 leader_election.cc:302] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 755c423f3ff545b99f6275c07b92a6eb; no voters:
I20260809 04:59:29.250052 21161 leader_election.cc:288] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:59:29.250200 21166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:59:29.251483 21166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 2 LEADER]: Becoming Leader. State: Replica: 755c423f3ff545b99f6275c07b92a6eb, State: Running, Role: LEADER
I20260809 04:59:29.252050 21166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:29.252463 21161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:59:29.254616 21171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 755c423f3ff545b99f6275c07b92a6eb. Latest consensus state: current_term: 2 leader_uuid: "755c423f3ff545b99f6275c07b92a6eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } } }
I20260809 04:59:29.254999 21171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:29.255548 21167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "755c423f3ff545b99f6275c07b92a6eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } } }
I20260809 04:59:29.255832 21167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:29.262766 21177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:59:29.275463 21177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:59:29.277077 21177 catalog_manager.cc:1259] Loaded cluster ID: 788ba0c50b4040b8937664e988a979ed
I20260809 04:59:29.277185 21177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:59:29.280493 21177 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 04:59:29.300174 21177 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 04:59:29.314466 21177 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:59:29.316362 21177 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Loaded TSK: 0
I20260809 04:59:29.318141 21177 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:59:29.581251 21163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:29.581537 21163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:29.581652 21163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:29.588912 21163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:29.589031 21163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:29.589187 21163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:59:29.598942 21163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:43573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=42127
--tserver_master_addrs=127.29.243.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:29.600313 21163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:29.601670 21163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:29.616698 21208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:29.616667 21211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:29.617300 21214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:29.618304 21163 server_base.cc:1034] running on GCE node
I20260809 04:59:29.618992 21163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:29.620043 21163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:29.621289 21163 hybrid_clock.cc:648] HybridClock initialized: now 1786251569621211 us; error 48 us; skew 500 ppm
I20260809 04:59:29.621675 21163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:29 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251569, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:29.625102 21163 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:59:29.626629 21163 webserver.cc:460] Webserver started at http://127.29.243.193:42127/ using document root <none> and password file <none>
I20260809 04:59:29.627388 21163 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:29.627521 21163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:29.633330 21163 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:59:29.636318 21225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:29.637861 21163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:59:29.638016 21163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "0fcf4105d19248f8be2da5daae36bff0"
format_stamp: "Formatted at 2026-08-09 04:59:26 on dist-test-slave-21b9"
server_key: "1af7a6671a8cdb83171f3ff2399ded85"
server_key_iv: "8ed40cd377d35ed1b3b03dfb63aa7b4f"
server_key_version: "encryptionkey@0"
I20260809 04:59:29.639016 21163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:29.697329 21163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:29.701889 21163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:29.702159 21163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:29.703192 21163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:29.704620 21163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:29.704728 21163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:29.704838 21163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:29.704909 21163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:29.735936 21163 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:43573
I20260809 04:59:29.735989 21400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:43573 every 8 connection(s)
I20260809 04:59:29.737426 21163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:59:29.747231 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 21163
I20260809 04:59:29.747967 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:33517
--local_ip_for_outbound_sockets=127.29.243.194
--tserver_master_addrs=127.29.243.254:33181
--webserver_port=39363
--webserver_interface=127.29.243.194
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:59:29 dist-test-slave-21b9 krb5kdc[20179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251569, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:29.761436 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:29.740245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:49297 (local address 127.29.243.254:33181)
0809 04:59:29.740804 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:29.740815 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:29.741426 (+ 611us) server_negotiation.cc:407] Connection header received
0809 04:59:29.743763 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:29.743778 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:29.744124 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:29.744417 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:29.745964 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:29.747531 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:29.749161 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:29.750102 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:29.750273 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:29.753706 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:29.753716 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:29.753734 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:29.753776 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:29.757223 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:29.757944 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:29.757952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:29.757959 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:29.758162 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:29.758539 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:29.758543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:29.758546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:29.759123 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:29.759290 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:29.760202 (+ 912us) server_negotiation.cc:299] Negotiation successful
0809 04:59:29.760511 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":241,"threads_started":1}
I20260809 04:59:29.761476 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:29.740436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:49297)
0809 04:59:29.741262 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:29.741316 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:29.743217 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:29.744600 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:29.744612 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:29.745197 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:29.745757 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:29.745775 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:29.747685 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:29.747689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:29.748984 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:29.748998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:29.750291 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:29.750295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:29.750485 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:29.751120 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:29.751143 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:29.753431 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:29.757362 (+ 3931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:29.757369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:29.757380 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:29.757815 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:29.758260 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:29.758263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:29.758266 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:29.758445 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:29.759257 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:29.759265 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:29.759719 (+ 454us) client_negotiation.cc:769] Sending connection context
0809 04:59:29.760021 (+ 302us) client_negotiation.cc:240] Negotiation successful
0809 04:59:29.760594 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":327,"threads_started":1}
I20260809 04:59:29.764619 21402 heartbeater.cc:347] Connected to a master server at 127.29.243.254:33181
I20260809 04:59:29.764966 21402 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:29.765847 21402 heartbeater.cc:511] Master 127.29.243.254:33181 requested a full tablet report, sending...
I20260809 04:59:29.768165 21106 ts_manager.cc:194] Registered new tserver with Master: 0fcf4105d19248f8be2da5daae36bff0 (127.29.243.193:43573)
I20260809 04:59:29.769635 21106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:49297
W20260809 04:59:30.093158 21408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:30.093436 21408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:30.093559 21408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:30.100692 21408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:30.100806 21408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:30.100961 21408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:59:30.110720 21408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:33517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=39363
--tserver_master_addrs=127.29.243.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:30.112051 21408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:30.113386 21408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:30.127172 21415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:30.127203 21416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:30.127148 21418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:30.128278 21408 server_base.cc:1034] running on GCE node
I20260809 04:59:30.128965 21408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:30.129942 21408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:30.131160 21408 hybrid_clock.cc:648] HybridClock initialized: now 1786251570131117 us; error 28 us; skew 500 ppm
I20260809 04:59:30.131640 21408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:30 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251570, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:30.134966 21408 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:59:30.136479 21408 webserver.cc:460] Webserver started at http://127.29.243.194:39363/ using document root <none> and password file <none>
I20260809 04:59:30.137143 21408 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:30.137256 21408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:30.142654 21408 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 04:59:30.145574 21428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:30.146919 21408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 04:59:30.147070 21408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60"
format_stamp: "Formatted at 2026-08-09 04:59:27 on dist-test-slave-21b9"
server_key: "ff99fd512104060da4c7020f0ed36c64"
server_key_iv: "3d81b99f94ab3c67bc428f4491dcfc53"
server_key_version: "encryptionkey@0"
I20260809 04:59:30.148125 21408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:30.163849 21408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:30.168195 21408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:30.168455 21408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:30.169489 21408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:30.170822 21408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:30.170928 21408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:30.171010 21408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:30.171067 21408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:30.209503 21408 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:33517
I20260809 04:59:30.209563 21599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:33517 every 8 connection(s)
I20260809 04:59:30.211088 21408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:59:30.219736 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 21408
I20260809 04:59:30.220366 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:45017
--local_ip_for_outbound_sockets=127.29.243.195
--tserver_master_addrs=127.29.243.254:33181
--webserver_port=35151
--webserver_interface=127.29.243.195
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:59:30 dist-test-slave-21b9 krb5kdc[20179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251570, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:30.232107 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.213726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:59791 (local address 127.29.243.254:33181)
0809 04:59:30.213942 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:30.213946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:30.215143 (+ 1197us) server_negotiation.cc:407] Connection header received
0809 04:59:30.217032 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:30.217038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:30.217146 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:30.217234 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:30.218765 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.219533 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.221175 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.221576 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.221686 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.225806 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:30.225813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:30.225817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:30.225857 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:30.228725 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:30.229672 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:30.229677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:30.229679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:30.229822 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:30.230274 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:30.230277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:30.230279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:30.230539 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:30.230647 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:30.231502 (+ 855us) server_negotiation.cc:299] Negotiation successful
0809 04:59:30.231742 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 04:59:30.233660 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.214201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:59791)
0809 04:59:30.214960 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:30.215007 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:30.216744 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:30.217500 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:30.217509 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:30.218021 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:30.218594 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:30.218611 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.219686 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.219692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:30.221030 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:30.221043 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.221797 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.221804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:30.222026 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.222738 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:30.222767 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:30.225543 (+ 2776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:30.228944 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:30.228957 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:30.228976 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:30.229556 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:30.229919 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:30.229923 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:30.229926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:30.230181 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:30.230651 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:30.230659 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:30.231104 (+ 445us) client_negotiation.cc:769] Sending connection context
0809 04:59:30.231429 (+ 325us) client_negotiation.cc:240] Negotiation successful
0809 04:59:30.232016 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":306,"threads_started":1}
I20260809 04:59:30.235095 21601 heartbeater.cc:347] Connected to a master server at 127.29.243.254:33181
I20260809 04:59:30.235430 21601 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:30.236191 21601 heartbeater.cc:511] Master 127.29.243.254:33181 requested a full tablet report, sending...
I20260809 04:59:30.237689 21106 ts_manager.cc:194] Registered new tserver with Master: 04c9fc9b2d96493b82b0dbd7c22c2d60 (127.29.243.194:33517)
I20260809 04:59:30.238503 21106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:59791
W20260809 04:59:30.560492 21605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:30.560755 21605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:30.560869 21605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:30.567927 21605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:30.568035 21605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:30.568156 21605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:59:30.577627 21605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:45017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=35151
--tserver_master_addrs=127.29.243.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:30.578881 21605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:30.580229 21605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:30.592980 21614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:30.592939 21619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:30.592909 21615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:30.593904 21605 server_base.cc:1034] running on GCE node
I20260809 04:59:30.594471 21605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:30.595360 21605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:30.596597 21605 hybrid_clock.cc:648] HybridClock initialized: now 1786251570596545 us; error 41 us; skew 500 ppm
I20260809 04:59:30.596930 21605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:30 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251570, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:30.599985 21605 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:59:30.601372 21605 webserver.cc:460] Webserver started at http://127.29.243.195:35151/ using document root <none> and password file <none>
I20260809 04:59:30.601995 21605 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:30.602118 21605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:30.607486 21605 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:59:30.610224 21630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:30.611621 21605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:59:30.611763 21605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "bb93c6824e914466bdc4ebdfbb6782bb"
format_stamp: "Formatted at 2026-08-09 04:59:27 on dist-test-slave-21b9"
server_key: "a100cb2fad3c4e76b18a8fa0d032c779"
server_key_iv: "cf0dc8a4d675de62375312f1c80b099f"
server_key_version: "encryptionkey@0"
I20260809 04:59:30.612670 21605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:30.654426 21605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:30.658484 21605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:30.658725 21605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:30.659772 21605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:30.661124 21605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:30.661221 21605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:30.661305 21605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:30.661360 21605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:30.691718 21605 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:45017
I20260809 04:59:30.691792 21806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:45017 every 8 connection(s)
I20260809 04:59:30.693199 21605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:59:30.699855 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 21605
Aug 09 04:59:30 dist-test-slave-21b9 krb5kdc[20179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251570, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:30.715126 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.695782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:42019 (local address 127.29.243.254:33181)
0809 04:59:30.695950 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:30.695954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:30.697097 (+ 1143us) server_negotiation.cc:407] Connection header received
0809 04:59:30.698853 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:30.698858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:30.698937 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:30.699022 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:30.700704 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.701615 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.703907 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.704636 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.704767 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.708693 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:30.708700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:30.708705 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:30.708744 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:30.711702 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:30.712553 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:30.712558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:30.712560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:30.712751 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:30.713244 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:30.713248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:30.713250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:30.713537 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:30.713651 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:30.714567 (+ 916us) server_negotiation.cc:299] Negotiation successful
0809 04:59:30.714772 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":52}
I20260809 04:59:30.716110 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.696191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:42019)
0809 04:59:30.696922 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:30.696970 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:30.698601 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:30.699228 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:30.699236 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:30.699814 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:30.700444 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:30.700473 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.701780 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.701784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:30.703758 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:30.703773 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.704845 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.704850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:30.705052 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.705845 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:30.705874 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:30.708474 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:30.711832 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:30.711842 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:30.711859 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:30.712420 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:30.712865 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:30.712869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:30.712873 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:30.713124 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:30.713677 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:30.713685 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:30.714110 (+ 425us) client_negotiation.cc:769] Sending connection context
0809 04:59:30.714424 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 04:59:30.714823 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":268,"threads_started":1}
I20260809 04:59:30.717242 21812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.700787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56574 (local address 127.29.243.254:33181)
0809 04:59:30.701238 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:30.701244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:30.701590 (+ 346us) server_negotiation.cc:407] Connection header received
0809 04:59:30.701717 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:30.701722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:30.701790 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:30.701868 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:30.703935 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.705077 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.706899 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.707662 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.707770 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.711376 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:30.711381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:30.711384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:30.711425 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:30.714329 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:30.715173 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:30.715179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:30.715183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:30.715347 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:30.715740 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:30.715745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:30.715748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:30.716068 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:30.716329 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:30.716693 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 04:59:30.716895 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":332,"thread_start_us":227,"threads_started":1}
I20260809 04:59:30.717803 21807 heartbeater.cc:347] Connected to a master server at 127.29.243.254:33181
I20260809 04:59:30.718112 21807 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:30.718894 21807 heartbeater.cc:511] Master 127.29.243.254:33181 requested a full tablet report, sending...
I20260809 04:59:30.720394 21106 ts_manager.cc:194] Registered new tserver with Master: bb93c6824e914466bdc4ebdfbb6782bb (127.29.243.195:45017)
I20260809 04:59:30.721205 21106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:42019
I20260809 04:59:30.731725 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:59:30.738334 21812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.732564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56580 (local address 127.29.243.254:33181)
0809 04:59:30.732752 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:30.732758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:30.733207 (+ 449us) server_negotiation.cc:407] Connection header received
0809 04:59:30.733403 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:30.733407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:30.733541 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:30.733668 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:59:30.734116 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.734804 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.736305 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.736710 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.736819 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.737328 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:59:30.737780 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:59:30.737917 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:30.738028 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 04:59:30.738107 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:59:30.739171 21106 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:56580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:59:30.746627 21106 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:59:30.773129 21402 heartbeater.cc:503] Master 127.29.243.254:33181 was elected leader, sending a full tablet report...
I20260809 04:59:30.779332 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.758646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:45017 (local address 127.0.0.1:50994)
0809 04:59:30.771367 (+ 12721us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:30.771384 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:30.771602 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:30.772444 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:30.772449 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:30.772474 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:30.772884 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:30.772896 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.774698 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.774703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:30.776486 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:30.776508 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.778201 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.778205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:30.778374 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.778434 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 04:59:30.778566 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 04:59:30.778751 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12559,"mutex_wait_us":69,"spinlock_wait_cycles":1792,"thread_start_us":9930,"threads_started":1}
I20260809 04:59:30.779572 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.759065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46050 (local address 127.29.243.194:33517)
0809 04:59:30.768142 (+ 9077us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:30.768151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:30.770780 (+ 2629us) server_negotiation.cc:407] Connection header received
0809 04:59:30.770868 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:30.770873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:30.771062 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:30.771261 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:30.772077 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.773728 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.776769 (+ 3041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.778186 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.778350 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.778512 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:30.779059 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 04:59:30.779215 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8908,"thread_start_us":8768,"threads_started":1}
I20260809 04:59:30.779886 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.759533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.29.243.195:45017)
0809 04:59:30.770893 (+ 11360us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:30.770900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:30.771481 (+ 581us) server_negotiation.cc:407] Connection header received
0809 04:59:30.771768 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:30.771775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:30.771985 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:30.772164 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:30.773089 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.774570 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.776678 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.777793 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.777918 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.778055 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:30.778950 (+ 895us) server_negotiation.cc:299] Negotiation successful
0809 04:59:30.779167 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11188,"thread_start_us":11026,"threads_started":1}
I20260809 04:59:30.779944 21826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.758483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:33517 (local address 127.0.0.1:46050)
0809 04:59:30.770505 (+ 12022us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:30.770526 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:30.770656 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:30.771552 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:30.771556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:30.771606 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:30.771937 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:30.771945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.773901 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.773907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:30.776588 (+ 2681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:30.776602 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.778413 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.778420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:30.778572 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.778601 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:59:30.778673 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 04:59:30.778775 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11881,"mutex_wait_us":137,"spinlock_wait_cycles":5888,"thread_start_us":11723,"threads_started":1}
I20260809 04:59:30.780032 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.758288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53918 (local address 127.29.243.193:43573)
0809 04:59:30.770393 (+ 12105us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:30.770402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:30.770446 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:59:30.770555 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:30.770564 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:30.770827 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:30.771039 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:30.771848 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.773582 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.777104 (+ 3522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.778481 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.778674 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.778835 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:30.779447 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 04:59:30.779611 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11929,"thread_start_us":11776,"threads_started":1}
I20260809 04:59:30.781276 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:30.758095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43573 (local address 127.0.0.1:53918)
0809 04:59:30.768113 (+ 10018us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:30.768156 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:30.768362 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:30.771068 (+ 2706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:30.771076 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:30.771111 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:30.771621 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:30.771634 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.773742 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.773747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:30.776925 (+ 3178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:30.776938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:30.778702 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:30.778709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:30.778865 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:30.778897 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:59:30.779047 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 04:59:30.779171 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9735,"mutex_wait_us":732,"thread_start_us":13229,"threads_started":1}
I20260809 04:59:30.783447 21495 tablet_service.cc:1467] Processing CreateTablet for tablet d137a6b1a35449d1827fe2bac4b1ec1f (DEFAULT_TABLE table=test-table [id=14d5bc52f9014383ba7fce2f3831c21f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:30.784284 21303 tablet_service.cc:1467] Processing CreateTablet for tablet d137a6b1a35449d1827fe2bac4b1ec1f (DEFAULT_TABLE table=test-table [id=14d5bc52f9014383ba7fce2f3831c21f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:30.785034 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d137a6b1a35449d1827fe2bac4b1ec1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:30.785503 21303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d137a6b1a35449d1827fe2bac4b1ec1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:30.785904 21705 tablet_service.cc:1467] Processing CreateTablet for tablet d137a6b1a35449d1827fe2bac4b1ec1f (DEFAULT_TABLE table=test-table [id=14d5bc52f9014383ba7fce2f3831c21f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:30.787297 21705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d137a6b1a35449d1827fe2bac4b1ec1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:30.797300 21835 tablet_bootstrap.cc:492] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: Bootstrap starting.
I20260809 04:59:30.800339 21836 tablet_bootstrap.cc:492] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: Bootstrap starting.
I20260809 04:59:30.801872 21835 tablet_bootstrap.cc:654] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:30.803179 21835 log.cc:824] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:30.803920 21836 tablet_bootstrap.cc:654] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:30.805177 21836 log.cc:824] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:30.805408 21838 tablet_bootstrap.cc:492] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: Bootstrap starting.
I20260809 04:59:30.806402 21835 tablet_bootstrap.cc:492] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: No bootstrap required, opened a new log
I20260809 04:59:30.806706 21835 ts_tablet_manager.cc:1397] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 04:59:30.808681 21836 tablet_bootstrap.cc:492] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: No bootstrap required, opened a new log
I20260809 04:59:30.808831 21838 tablet_bootstrap.cc:654] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:30.808951 21836 ts_tablet_manager.cc:1397] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260809 04:59:30.809777 21838 log.cc:824] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:30.812252 21838 tablet_bootstrap.cc:492] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: No bootstrap required, opened a new log
I20260809 04:59:30.812482 21838 ts_tablet_manager.cc:1397] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260809 04:59:30.816956 21835 raft_consensus.cc:348] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } }
I20260809 04:59:30.817394 21835 raft_consensus.cc:374] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:30.817513 21835 raft_consensus.cc:729] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fcf4105d19248f8be2da5daae36bff0, State: Initialized, Role: FOLLOWER
I20260809 04:59:30.818233 21835 consensus_queue.cc:260] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } }
I20260809 04:59:30.819006 21836 raft_consensus.cc:348] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } }
I20260809 04:59:30.819437 21836 raft_consensus.cc:374] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:30.819535 21836 raft_consensus.cc:729] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c9fc9b2d96493b82b0dbd7c22c2d60, State: Initialized, Role: FOLLOWER
I20260809 04:59:30.820106 21835 ts_tablet_manager.cc:1428] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: Time spent starting tablet: real 0.013s user 0.008s sys 0.006s
I20260809 04:59:30.820013 21838 raft_consensus.cc:348] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } }
I20260809 04:59:30.820386 21838 raft_consensus.cc:374] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:30.820226 21836 consensus_queue.cc:260] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } }
I20260809 04:59:30.820482 21838 raft_consensus.cc:729] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb93c6824e914466bdc4ebdfbb6782bb, State: Initialized, Role: FOLLOWER
I20260809 04:59:30.821306 21601 heartbeater.cc:503] Master 127.29.243.254:33181 was elected leader, sending a full tablet report...
I20260809 04:59:30.821148 21838 consensus_queue.cc:260] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } }
I20260809 04:59:30.821954 21836 ts_tablet_manager.cc:1428] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20260809 04:59:30.822907 21807 heartbeater.cc:503] Master 127.29.243.254:33181 was elected leader, sending a full tablet report...
I20260809 04:59:30.823695 21838 ts_tablet_manager.cc:1428] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
W20260809 04:59:30.944803 21809 tablet.cc:2369] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:59:30.963080 21603 tablet.cc:2369] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:59:30.989794 21403 tablet.cc:2369] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:59:31.172300 21844 raft_consensus.cc:482] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:59:31.172667 21844 raft_consensus.cc:504] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } }
I20260809 04:59:31.174510 21844 leader_election.cc:288] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04c9fc9b2d96493b82b0dbd7c22c2d60 (127.29.243.194:33517), bb93c6824e914466bdc4ebdfbb6782bb (127.29.243.195:45017)
I20260809 04:59:31.181190 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:31.175156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:53475 (local address 127.29.243.195:45017)
0809 04:59:31.175397 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:31.175404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:31.176076 (+ 672us) server_negotiation.cc:407] Connection header received
0809 04:59:31.176241 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:31.176244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:31.176306 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:31.176382 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:31.176993 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.177696 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.179522 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.180293 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.180447 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:31.180522 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:31.180821 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 04:59:31.180946 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 04:59:31.181190 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:31.175024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:45981 (local address 127.29.243.194:33517)
0809 04:59:31.175252 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:31.175258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:31.175943 (+ 685us) server_negotiation.cc:407] Connection header received
0809 04:59:31.176177 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:31.176182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:31.176244 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:31.176322 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:31.176911 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.177625 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.179503 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.180256 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.180402 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:31.180491 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:31.180815 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 04:59:31.180943 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 04:59:31.181604 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:31.175042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:45017 (local address 127.29.243.193:53475)
0809 04:59:31.175963 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:31.175978 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:31.176106 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:31.176409 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:31.176413 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:31.176440 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:31.176826 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:31.176836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.177827 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.177833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:31.179377 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:31.179390 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.180516 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.180522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:31.180675 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:31.180698 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:59:31.180773 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 04:59:31.180907 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":58,"thread_start_us":442,"threads_started":1}
I20260809 04:59:31.181635 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:31.174971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:33517 (local address 127.29.243.193:45981)
0809 04:59:31.175827 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:31.175844 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:31.176026 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:31.176355 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:31.176359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:31.176383 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:31.176763 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:31.176774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.177753 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.177759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:31.179342 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:31.179354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.180478 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.180485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:31.180623 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:31.180686 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 04:59:31.180767 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 04:59:31.180895 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":49,"thread_start_us":649,"threads_started":1}
I20260809 04:59:31.182789 21529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d137a6b1a35449d1827fe2bac4b1ec1f" candidate_uuid: "0fcf4105d19248f8be2da5daae36bff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" is_pre_election: true
I20260809 04:59:31.182789 21736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d137a6b1a35449d1827fe2bac4b1ec1f" candidate_uuid: "0fcf4105d19248f8be2da5daae36bff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" is_pre_election: true
I20260809 04:59:31.183459 21736 raft_consensus.cc:2454] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fcf4105d19248f8be2da5daae36bff0 in term 0.
I20260809 04:59:31.183458 21529 raft_consensus.cc:2454] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fcf4105d19248f8be2da5daae36bff0 in term 0.
I20260809 04:59:31.184198 21228 leader_election.cc:302] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c9fc9b2d96493b82b0dbd7c22c2d60, 0fcf4105d19248f8be2da5daae36bff0; no voters:
I20260809 04:59:31.184657 21844 raft_consensus.cc:2781] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:59:31.184801 21844 raft_consensus.cc:482] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:59:31.184903 21844 raft_consensus.cc:3037] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:31.187206 21844 raft_consensus.cc:504] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } }
I20260809 04:59:31.187834 21844 leader_election.cc:288] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [CANDIDATE]: Term 1 election: Requested vote from peers 04c9fc9b2d96493b82b0dbd7c22c2d60 (127.29.243.194:33517), bb93c6824e914466bdc4ebdfbb6782bb (127.29.243.195:45017)
I20260809 04:59:31.188309 21529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d137a6b1a35449d1827fe2bac4b1ec1f" candidate_uuid: "0fcf4105d19248f8be2da5daae36bff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60"
I20260809 04:59:31.188331 21736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d137a6b1a35449d1827fe2bac4b1ec1f" candidate_uuid: "0fcf4105d19248f8be2da5daae36bff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb93c6824e914466bdc4ebdfbb6782bb"
I20260809 04:59:31.188532 21736 raft_consensus.cc:3037] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:31.188525 21529 raft_consensus.cc:3037] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:31.191645 21736 raft_consensus.cc:2454] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fcf4105d19248f8be2da5daae36bff0 in term 1.
I20260809 04:59:31.191645 21529 raft_consensus.cc:2454] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fcf4105d19248f8be2da5daae36bff0 in term 1.
I20260809 04:59:31.192072 21228 leader_election.cc:302] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c9fc9b2d96493b82b0dbd7c22c2d60, 0fcf4105d19248f8be2da5daae36bff0; no voters:
I20260809 04:59:31.192440 21844 raft_consensus.cc:2781] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:59:31.192875 21844 raft_consensus.cc:686] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 1 LEADER]: Becoming Leader. State: Replica: 0fcf4105d19248f8be2da5daae36bff0, State: Running, Role: LEADER
I20260809 04:59:31.193397 21844 consensus_queue.cc:237] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } }
I20260809 04:59:31.198163 21102 catalog_manager.cc:5495] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fcf4105d19248f8be2da5daae36bff0 (127.29.243.193). New cstate: current_term: 1 leader_uuid: "0fcf4105d19248f8be2da5daae36bff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcf4105d19248f8be2da5daae36bff0" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 } health_report { overall_health: UNKNOWN } } }
I20260809 04:59:31.233475 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:31.228047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53924 (local address 127.29.243.193:43573)
0809 04:59:31.228244 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:31.228249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:31.228267 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:59:31.228434 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:31.228467 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:31.228551 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:31.228693 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:59:31.229039 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.229781 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.231247 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.231759 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.231852 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:31.232376 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:59:31.232885 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:59:31.232999 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:31.233127 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 04:59:31.233194 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:59:31.241918 21736 raft_consensus.cc:1261] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb [term 1 FOLLOWER]: Refusing update from remote peer 0fcf4105d19248f8be2da5daae36bff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:59:31.241910 21529 raft_consensus.cc:1261] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 1 FOLLOWER]: Refusing update from remote peer 0fcf4105d19248f8be2da5daae36bff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:59:31.242915 21857 consensus_queue.cc:1035] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb93c6824e914466bdc4ebdfbb6782bb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 45017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:59:31.243290 21844 consensus_queue.cc:1035] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 33517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:59:31.256492 21863 mvcc.cc:205] Tried to move back new op lower bound from 7316486435797188608 to 7316486435611860992. Current Snapshot: MvccSnapshot[applied={T|T < 7316486435797188608}]
I20260809 04:59:31.256978 21861 mvcc.cc:205] Tried to move back new op lower bound from 7316486435797188608 to 7316486435611860992. Current Snapshot: MvccSnapshot[applied={T|T < 7316486435797188608}]
I20260809 04:59:31.271664 21102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:56580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:59:31.271932 21102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:56580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:59:31.275136 21102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Sending DeleteTablet for 3 replicas of tablet d137a6b1a35449d1827fe2bac4b1ec1f
I20260809 04:59:31.276320 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet d137a6b1a35449d1827fe2bac4b1ec1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:31 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:46050
I20260809 04:59:31.276292 21303 tablet_service.cc:1514] Processing DeleteTablet for tablet d137a6b1a35449d1827fe2bac4b1ec1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:31 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:53918
I20260809 04:59:31.277204 21876 tablet_replica.cc:330] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: stopping tablet replica
I20260809 04:59:31.277293 21705 tablet_service.cc:1514] Processing DeleteTablet for tablet d137a6b1a35449d1827fe2bac4b1ec1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:31 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:50994
I20260809 04:59:31.277799 21876 raft_consensus.cc:2229] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:59:31.278405 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 21163
I20260809 04:59:31.278394 21876 raft_consensus.cc:2258] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:31.281533 21876 ts_tablet_manager.cc:1905] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:31.289354 21876 ts_tablet_manager.cc:1918] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:59:31.289625 21877 tablet_replica.cc:330] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: stopping tablet replica
I20260809 04:59:31.290263 21877 raft_consensus.cc:2229] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:59:31.290592 21879 tablet_replica.cc:330] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: stopping tablet replica
I20260809 04:59:31.291041 21879 raft_consensus.cc:2229] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:59:31.291426 21877 raft_consensus.cc:2258] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:31.291604 21879 raft_consensus.cc:2258] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:31.292285 21876 log.cc:1196] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/d137a6b1a35449d1827fe2bac4b1ec1f
I20260809 04:59:31.292888 21876 ts_tablet_manager.cc:1939] T d137a6b1a35449d1827fe2bac4b1ec1f P 04c9fc9b2d96493b82b0dbd7c22c2d60: Deleting consensus metadata
I20260809 04:59:31.294267 21877 ts_tablet_manager.cc:1905] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:31.294920 21085 catalog_manager.cc:4841] TS 04c9fc9b2d96493b82b0dbd7c22c2d60 (127.29.243.194:33517): tablet d137a6b1a35449d1827fe2bac4b1ec1f (table test-table [id=14d5bc52f9014383ba7fce2f3831c21f]) successfully deleted
I20260809 04:59:31.296801 21879 ts_tablet_manager.cc:1905] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:31.302129 21877 ts_tablet_manager.cc:1918] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:59:31.302395 21877 log.cc:1196] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/d137a6b1a35449d1827fe2bac4b1ec1f
I20260809 04:59:31.303109 21877 ts_tablet_manager.cc:1939] T d137a6b1a35449d1827fe2bac4b1ec1f P 0fcf4105d19248f8be2da5daae36bff0: Deleting consensus metadata
I20260809 04:59:31.305491 21087 catalog_manager.cc:4841] TS 0fcf4105d19248f8be2da5daae36bff0 (127.29.243.193:43573): tablet d137a6b1a35449d1827fe2bac4b1ec1f (table test-table [id=14d5bc52f9014383ba7fce2f3831c21f]) successfully deleted
I20260809 04:59:31.306831 21879 ts_tablet_manager.cc:1918] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:59:31.307056 21879 log.cc:1196] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/d137a6b1a35449d1827fe2bac4b1ec1f
I20260809 04:59:31.307673 21879 ts_tablet_manager.cc:1939] T d137a6b1a35449d1827fe2bac4b1ec1f P bb93c6824e914466bdc4ebdfbb6782bb: Deleting consensus metadata
I20260809 04:59:31.309176 21088 catalog_manager.cc:4841] TS bb93c6824e914466bdc4ebdfbb6782bb (127.29.243.195:45017): tablet d137a6b1a35449d1827fe2bac4b1ec1f (table test-table [id=14d5bc52f9014383ba7fce2f3831c21f]) successfully deleted
I20260809 04:59:31.312551 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:31.280025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53940 (local address 127.29.243.193:43573)
0809 04:59:31.280206 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:31.280212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:31.294664 (+ 14452us) server_negotiation.cc:407] Connection header received
0809 04:59:31.295619 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:31.295625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:31.295761 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:31.295884 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:31.297835 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.299076 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.301159 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.301854 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.301992 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:31.305518 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:31.305532 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:31.305551 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:31.305590 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:31.309609 (+ 4019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:31.310289 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:31.310299 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:31.310303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:31.310483 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:31.310828 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:31.310833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:31.310837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:31.311347 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:31.311656 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:31.311930 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 04:59:31.312175 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:59:31.313362 21394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:31.413069 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 21163
I20260809 04:59:31.441980 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 21408
I20260809 04:59:31.456036 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:31.442680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.29.243.194:33517)
0809 04:59:31.442840 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:31.442844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:31.442898 (+ 54us) server_negotiation.cc:407] Connection header received
0809 04:59:31.443069 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:31.443073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:31.443152 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:31.443230 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:31.444742 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.445502 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.446839 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.447227 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.447540 (+ 313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:31.450112 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:31.450128 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:31.450145 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:31.450177 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:31.453191 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:31.453966 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:31.453974 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:31.453976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:31.454111 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:31.454503 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:31.454506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:31.454508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:31.454883 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:31.455109 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:31.455590 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 04:59:31.455766 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:59:31.456754 21594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:31.545507 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 21408
I20260809 04:59:31.572739 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 21605
I20260809 04:59:31.585968 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:31.573575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.29.243.195:45017)
0809 04:59:31.573750 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:31.573755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:31.573811 (+ 56us) server_negotiation.cc:407] Connection header received
0809 04:59:31.573962 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:31.573966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:31.574055 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:31.574139 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:31.575632 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.576348 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.577639 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:31.578022 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:31.578116 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:31.580615 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:31.580632 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:31.580651 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:31.580688 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:31.583570 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:31.584249 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:31.584257 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:31.584259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:31.584410 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:31.584722 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:31.584725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:31.584727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:31.585088 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:31.585298 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:31.585538 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 04:59:31.585700 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:59:31.586627 21798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:31.674494 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 21605
I20260809 04:59:31.698238 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 21053
I20260809 04:59:31.699409 21153 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:31.812444 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 21053
I20260809 04:59:31.841765 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:33181
--webserver_interface=127.29.243.254
--webserver_port=37933
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:33181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:32.185119 21885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:32.185395 21885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:32.185485 21885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:32.192484 21885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:59:32.192595 21885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:32.192641 21885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:32.192699 21885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:59:32.192742 21885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:59:32.202221 21885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:33181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:33181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=37933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:32.203480 21885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:32.204762 21885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:32.216002 21898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:32.215973 21896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:32.216288 21901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:32.216796 21885 server_base.cc:1034] running on GCE node
I20260809 04:59:32.217540 21885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:32.218905 21885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:32.220103 21885 hybrid_clock.cc:648] HybridClock initialized: now 1786251572220053 us; error 36 us; skew 500 ppm
I20260809 04:59:32.220444 21885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:32 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251572, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:32.223582 21885 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:59:32.225117 21885 webserver.cc:460] Webserver started at http://127.29.243.254:37933/ using document root <none> and password file <none>
I20260809 04:59:32.225780 21885 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:32.225883 21885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:32.231387 21885 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:59:32.234432 21909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:32.235836 21885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:59:32.236017 21885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "755c423f3ff545b99f6275c07b92a6eb"
format_stamp: "Formatted at 2026-08-09 04:59:25 on dist-test-slave-21b9"
server_key: "292a056e9843b226418cf3bb0d342d6e"
server_key_iv: "5a45917c04d6be601bdff82025e7185e"
server_key_version: "encryptionkey@0"
I20260809 04:59:32.236975 21885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:32.261307 21885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:32.265478 21885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:32.265731 21885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:32.283537 21885 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:33181
I20260809 04:59:32.283558 21991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:33181 every 8 connection(s)
I20260809 04:59:32.285033 21885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:59:32.288946 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 21885
I20260809 04:59:32.289765 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:43573
--local_ip_for_outbound_sockets=127.29.243.193
--tserver_master_addrs=127.29.243.254:33181
--webserver_port=42127
--webserver_interface=127.29.243.193
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:59:32.292261 21992 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:59:32.295527 21992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Bootstrap starting.
I20260809 04:59:32.316718 21992 log.cc:824] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:32.333141 21992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:59:32.333767 21992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb: Bootstrap complete.
I20260809 04:59:32.342906 21992 raft_consensus.cc:348] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:32.343865 21992 raft_consensus.cc:729] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755c423f3ff545b99f6275c07b92a6eb, State: Initialized, Role: FOLLOWER
I20260809 04:59:32.344456 21992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:32.344697 21992 raft_consensus.cc:388] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:59:32.344812 21992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:59:32.344972 21992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 2 FOLLOWER]: Advancing to term 3
I20260809 04:59:32.348901 21992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:32.349413 21992 leader_election.cc:302] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 755c423f3ff545b99f6275c07b92a6eb; no voters:
I20260809 04:59:32.350076 21992 leader_election.cc:288] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 04:59:32.350224 22000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 3 FOLLOWER]: Leader election won for term 3
I20260809 04:59:32.351432 22000 raft_consensus.cc:686] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [term 3 LEADER]: Becoming Leader. State: Replica: 755c423f3ff545b99f6275c07b92a6eb, State: Running, Role: LEADER
I20260809 04:59:32.351986 22000 consensus_queue.cc:237] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } }
I20260809 04:59:32.352339 21992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:59:32.354823 22007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 755c423f3ff545b99f6275c07b92a6eb. Latest consensus state: current_term: 3 leader_uuid: "755c423f3ff545b99f6275c07b92a6eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } } }
I20260809 04:59:32.355201 22007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:32.354796 22001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "755c423f3ff545b99f6275c07b92a6eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755c423f3ff545b99f6275c07b92a6eb" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 33181 } } }
I20260809 04:59:32.355986 22001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755c423f3ff545b99f6275c07b92a6eb [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:32.367718 22017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:59:32.379339 22017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:59:32.381007 22017 catalog_manager.cc:1259] Loaded cluster ID: 788ba0c50b4040b8937664e988a979ed
I20260809 04:59:32.381135 22017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 04:59:32.383385 22017 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f27251819ad google::LogMessage::Fail() at ??:0
@ 0x7f272518488a google::LogMessage::SendToLog() at ??:0
@ 0x7f27251814de google::LogMessage::Flush() at ??:0
@ 0x7f2725182a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f273abb4512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f273abb1976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f273ac39610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f273ac39439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2727dcd345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2727dd091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f2727dd0799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2727dabeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f272b0856db start_thread at ??:0
@ 0x7f272401b61f clone at ??:0
W20260809 04:59:32.699509 21998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:32.699803 21998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:32.699944 21998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:32.707212 21998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:32.707365 21998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:32.707518 21998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:59:32.717685 21998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:43573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=42127
--tserver_master_addrs=127.29.243.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:32.719135 21998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:32.720510 21998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:32.733776 22065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:32.733824 22063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:32.733851 22062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:32.734879 21998 server_base.cc:1034] running on GCE node
I20260809 04:59:32.735581 21998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:32.736735 21998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:32.737960 21998 hybrid_clock.cc:648] HybridClock initialized: now 1786251572737901 us; error 48 us; skew 500 ppm
I20260809 04:59:32.738273 21998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:32 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251572, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:32.741453 21998 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:59:32.742967 21998 webserver.cc:460] Webserver started at http://127.29.243.193:42127/ using document root <none> and password file <none>
I20260809 04:59:32.743701 21998 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:32.743827 21998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:32.749197 21998 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:59:32.751936 22076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:32.753330 21998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 04:59:32.753497 21998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "0fcf4105d19248f8be2da5daae36bff0"
format_stamp: "Formatted at 2026-08-09 04:59:26 on dist-test-slave-21b9"
server_key: "1af7a6671a8cdb83171f3ff2399ded85"
server_key_iv: "8ed40cd377d35ed1b3b03dfb63aa7b4f"
server_key_version: "encryptionkey@0"
I20260809 04:59:32.754489 21998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:32.764889 21998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:32.769009 21998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:32.769273 21998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:32.770232 21998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:32.771615 21998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:32.771721 21998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:32.771829 21998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:32.771901 21998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:32.802965 21998 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:43573
I20260809 04:59:32.803048 22262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:43573 every 8 connection(s)
I20260809 04:59:32.804498 21998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:59:32.809674 22268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:32.807737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:39125)
0809 04:59:32.808500 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:32.808705 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":284,"threads_started":1}
I20260809 04:59:32.810081 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 21998
W20260809 04:59:32.810278 22263 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:33181 (0 consecutive failures): Network error: Failed to ping master at 127.29.243.254:33181: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
I20260809 04:59:32.810834 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:33517
--local_ip_for_outbound_sockets=127.29.243.194
--tserver_master_addrs=127.29.243.254:33181
--webserver_port=39363
--webserver_interface=127.29.243.194
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:59:32.811375 22268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:32.810859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:34167)
0809 04:59:32.811080 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:32.811173 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 04:59:32.812302 22268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:32.811940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:52751)
0809 04:59:32.812100 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:32.812185 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 04:59:32.812606 22263 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:59:33.147982 22269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:33.148269 22269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:33.148373 22269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:33.155509 22269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:33.155619 22269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:33.155737 22269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:59:33.165393 22269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:33517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=39363
--tserver_master_addrs=127.29.243.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:33.166683 22269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:33.168057 22269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:33.181255 22282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:33.181234 22280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:33.181449 22279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:33.182052 22269 server_base.cc:1034] running on GCE node
I20260809 04:59:33.182642 22269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:33.183615 22269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:33.184881 22269 hybrid_clock.cc:648] HybridClock initialized: now 1786251573184810 us; error 47 us; skew 500 ppm
I20260809 04:59:33.185246 22269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:33 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251573, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:33.188583 22269 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:59:33.190099 22269 webserver.cc:460] Webserver started at http://127.29.243.194:39363/ using document root <none> and password file <none>
I20260809 04:59:33.190819 22269 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:33.190930 22269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:33.196897 22269 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 04:59:33.199960 22291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:33.201333 22269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:59:33.201517 22269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "04c9fc9b2d96493b82b0dbd7c22c2d60"
format_stamp: "Formatted at 2026-08-09 04:59:27 on dist-test-slave-21b9"
server_key: "ff99fd512104060da4c7020f0ed36c64"
server_key_iv: "3d81b99f94ab3c67bc428f4491dcfc53"
server_key_version: "encryptionkey@0"
I20260809 04:59:33.202517 22269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:33.217201 22269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:33.221385 22269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:33.221647 22269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:33.222645 22269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:33.224006 22269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:33.224103 22269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:33.224211 22269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:33.224284 22269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:33.257149 22269 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:33517
I20260809 04:59:33.257216 22485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:33517 every 8 connection(s)
I20260809 04:59:33.258699 22269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:59:33.263732 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:33.261712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:45193)
0809 04:59:33.262542 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:33.262759 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":305,"threads_started":1}
W20260809 04:59:33.264403 22487 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:33181 (0 consecutive failures): Network error: Failed to ping master at 127.29.243.254:33181: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
I20260809 04:59:33.265228 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:33.264913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:49337)
0809 04:59:33.265062 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:33.265127 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 04:59:33.266029 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:33.265720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:43419)
0809 04:59:33.265852 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:33.265925 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 04:59:33.266252 22487 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 04:59:33.268915 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 22269
I20260809 04:59:33.269497 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:45017
--local_ip_for_outbound_sockets=127.29.243.195
--tserver_master_addrs=127.29.243.254:33181
--webserver_port=35151
--webserver_interface=127.29.243.195
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:33.608366 22490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:33.608655 22490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:33.608779 22490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:33.615837 22490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:33.615962 22490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:33.616117 22490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:59:33.625666 22490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:45017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=35151
--tserver_master_addrs=127.29.243.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:33.627032 22490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:33.628387 22490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:33.641423 22505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:33.641458 22506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:33.641499 22508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:33.642498 22490 server_base.cc:1034] running on GCE node
I20260809 04:59:33.643110 22490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:33.644111 22490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:33.645326 22490 hybrid_clock.cc:648] HybridClock initialized: now 1786251573645275 us; error 44 us; skew 500 ppm
I20260809 04:59:33.645654 22490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:33 dist-test-slave-21b9 krb5kdc[20179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251573, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:33.648764 22490 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:59:33.650209 22490 webserver.cc:460] Webserver started at http://127.29.243.195:35151/ using document root <none> and password file <none>
I20260809 04:59:33.650864 22490 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:33.650965 22490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:33.656298 22490 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 04:59:33.659196 22518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:33.660683 22490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:59:33.660825 22490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "bb93c6824e914466bdc4ebdfbb6782bb"
format_stamp: "Formatted at 2026-08-09 04:59:27 on dist-test-slave-21b9"
server_key: "a100cb2fad3c4e76b18a8fa0d032c779"
server_key_iv: "cf0dc8a4d675de62375312f1c80b099f"
server_key_version: "encryptionkey@0"
I20260809 04:59:33.661762 22490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:33.688512 22490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:33.692701 22490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:33.692958 22490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:33.693921 22490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:33.695232 22490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:33.695390 22490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:33.695499 22490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:33.695577 22490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:33.726784 22490 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:45017
I20260809 04:59:33.726856 22702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:45017 every 8 connection(s)
I20260809 04:59:33.728228 22490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:59:33.733027 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:33.731127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:34001)
0809 04:59:33.731887 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:33.732124 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":315,"threads_started":1}
W20260809 04:59:33.733673 22703 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:33181 (0 consecutive failures): Network error: Failed to ping master at 127.29.243.254:33181: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
I20260809 04:59:33.734419 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:33.734117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:37451)
0809 04:59:33.734253 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:33.734319 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 04:59:33.735163 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:33.734890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:34883)
0809 04:59:33.734992 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:33.735066 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 04:59:33.735395 22703 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 04:59:33.736999 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 22490
I20260809 04:59:33.814018 22710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:33.813160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:47175)
0809 04:59:33.813655 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:33.813778 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":248,"threads_started":1}
I20260809 04:59:34.267696 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:34.266809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:34097)
0809 04:59:34.267311 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:34.267469 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":223,"threads_started":1}
I20260809 04:59:34.736646 22713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:34.735872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:39007)
0809 04:59:34.736315 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:34.736426 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":204,"threads_started":1}
I20260809 04:59:34.815608 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:34.814870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:33731)
0809 04:59:34.815345 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:34.815429 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":217,"threads_started":1}
I20260809 04:59:35.269292 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:35.268553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:42495)
0809 04:59:35.269020 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:35.269120 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":224,"threads_started":1}
I20260809 04:59:35.738371 22719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:35.737565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:40811)
0809 04:59:35.738084 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:35.738173 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":240,"threads_started":1}
I20260809 04:59:35.817103 22722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:35.816365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:49969)
0809 04:59:35.816836 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:35.816928 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":250,"threads_started":1}
I20260809 04:59:36.270886 22725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:36.270162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:48009)
0809 04:59:36.270609 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:36.270694 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":231,"threads_started":1}
I20260809 04:59:36.739982 22726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:36.739230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:37499)
0809 04:59:36.739706 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:36.739787 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":238,"threads_started":1}
I20260809 04:59:36.818745 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:36.817977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:55735)
0809 04:59:36.818457 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:36.818543 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":225,"threads_started":1}
I20260809 04:59:37.272481 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:37.271717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:43625)
0809 04:59:37.272189 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:37.272275 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":229,"threads_started":1}
I20260809 04:59:37.741616 22729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:37.740859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:50773)
0809 04:59:37.741330 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:37.741417 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":237,"threads_started":1}
I20260809 04:59:37.820384 22731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:37.819683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:41419)
0809 04:59:37.820128 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:37.820210 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":216,"threads_started":1}
I20260809 04:59:38.274188 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:38.273329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:58435)
0809 04:59:38.273920 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:38.274008 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":326,"threads_started":1}
I20260809 04:59:38.743394 22736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:38.742543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:33359)
0809 04:59:38.743080 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:38.743164 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":286,"threads_started":1}
I20260809 04:59:38.822005 22738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:38.821255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:55575)
0809 04:59:38.821725 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:38.821820 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":245,"threads_started":1}
I20260809 04:59:39.279968 22739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:39.275083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:50319)
0809 04:59:39.279687 (+ 4604us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:39.279787 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4490,"thread_start_us":4367,"threads_started":1}
I20260809 04:59:39.744974 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:39.744248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:52617)
0809 04:59:39.744721 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:39.744812 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":253,"threads_started":1}
I20260809 04:59:39.823740 22742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:39.822916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:40623)
0809 04:59:39.823428 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:39.823513 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":212,"threads_started":1}
I20260809 04:59:40.281864 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:40.281031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:46359)
0809 04:59:40.281550 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:40.281638 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":213,"threads_started":1}
I20260809 04:59:40.746630 22745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:40.745788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:60757)
0809 04:59:40.746310 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:40.746388 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":236,"threads_started":1}
I20260809 04:59:40.825431 22746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:40.824624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:43111)
0809 04:59:40.825147 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:40.825238 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":288,"threads_started":1}
I20260809 04:59:41.283593 22747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:41.282781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:56373)
0809 04:59:41.283293 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:41.283412 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":253,"threads_started":1}
I20260809 04:59:41.748260 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:41.747483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:53111)
0809 04:59:41.747999 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:41.748091 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":244,"threads_started":1}
I20260809 04:59:41.827222 22749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:41.826451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:43191)
0809 04:59:41.826949 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:41.827040 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":254,"threads_started":1}
I20260809 04:59:42.285351 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:42.284567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:38857)
0809 04:59:42.285087 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:42.285171 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":255,"threads_started":1}
I20260809 04:59:42.749886 22752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:42.749134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:43741)
0809 04:59:42.749625 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:42.749706 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":234,"threads_started":1}
I20260809 04:59:42.829089 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:42.828288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:57839)
0809 04:59:42.828795 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:42.828887 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":284,"threads_started":1}
I20260809 04:59:43.287160 22754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:43.286253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:46249)
0809 04:59:43.286816 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:43.286924 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":309,"threads_started":1}
I20260809 04:59:43.751644 22756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:43.750840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:42017)
0809 04:59:43.751342 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:43.751476 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":253,"threads_started":1}
I20260809 04:59:43.831022 22758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:43.830010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:49479)
0809 04:59:43.830685 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:43.830797 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":329,"threads_started":1}
I20260809 04:59:44.289660 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:44.288725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:55835)
0809 04:59:44.289293 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:44.289413 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":284,"threads_started":1}
I20260809 04:59:44.753444 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:44.752641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:60849)
0809 04:59:44.753149 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:44.753252 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":263,"threads_started":1}
I20260809 04:59:44.832724 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:44.831939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:37897)
0809 04:59:44.832436 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:44.832528 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":245,"threads_started":1}
I20260809 04:59:45.291483 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:45.290674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:48175)
0809 04:59:45.291174 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:45.291262 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":253,"threads_started":1}
I20260809 04:59:45.755324 22767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:45.754401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:51449)
0809 04:59:45.754958 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:45.755092 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":298,"threads_started":1}
I20260809 04:59:45.834434 22768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:45.833617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:47273)
0809 04:59:45.834136 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:45.834253 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":244,"threads_started":1}
I20260809 04:59:46.293339 22769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:46.292550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:33353)
0809 04:59:46.293049 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:46.293147 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":237,"threads_started":1}
I20260809 04:59:46.757011 22770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:46.756245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:43163)
0809 04:59:46.756711 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:46.756805 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":219,"threads_started":1}
I20260809 04:59:46.836229 22771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:46.835421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:55755)
0809 04:59:46.835922 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:46.836015 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":248,"threads_started":1}
I20260809 04:59:47.295015 22774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:47.294249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:55667)
0809 04:59:47.294731 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:47.294829 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":240,"threads_started":1}
I20260809 04:59:47.758632 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:47.757918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:50191)
0809 04:59:47.758355 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:47.758440 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":216,"threads_started":1}
I20260809 04:59:47.838150 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:47.837377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.193:38151)
0809 04:59:47.837833 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:47.837957 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":241,"threads_started":1}
I20260809 04:59:48.296868 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:48.296147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.194:41869)
0809 04:59:48.296590 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:48.296687 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":236,"threads_started":1}
I20260809 04:59:48.748948 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 21998
I20260809 04:59:48.760525 22782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:48.759713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:33181 (local address 127.29.243.195:35547)
0809 04:59:48.760180 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:48.760295 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":238,"threads_started":1}
I20260809 04:59:48.771147 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:48.750126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.29.243.193:43573)
0809 04:59:48.750901 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:48.750922 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:48.750973 (+ 51us) server_negotiation.cc:407] Connection header received
0809 04:59:48.752604 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:48.752615 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:48.753071 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:48.753413 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:48.754976 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:48.756254 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:48.757526 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:48.758189 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:48.758363 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:48.765014 (+ 6651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:48.765026 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:48.765047 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:48.765087 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:48.767906 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:48.768587 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:48.768594 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:48.768601 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:48.768770 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:48.769168 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:48.769172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:48.769174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:48.769689 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:48.769987 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:48.770450 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 04:59:48.770856 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":353,"threads_started":1}
I20260809 04:59:48.772668 22258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:48.855544 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 21998
I20260809 04:59:48.883706 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 22269
I20260809 04:59:48.901556 22787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:48.884711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40136 (local address 127.29.243.194:33517)
0809 04:59:48.885250 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:48.885266 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:48.885307 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:59:48.886899 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:48.886909 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:48.887392 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:48.887735 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:48.889277 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:48.890522 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:48.891908 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:48.892631 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:48.892813 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:48.895428 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:48.895437 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:48.895454 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:48.895493 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:48.898266 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:48.898951 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:48.898959 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:48.898966 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:48.899152 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:48.899550 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:48.899556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:48.899559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:48.900092 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:48.900384 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:48.900851 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 04:59:48.901270 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":236,"threads_started":1}
I20260809 04:59:48.902972 22479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:48.984704 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 22269
I20260809 04:59:49.013216 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 22490
I20260809 04:59:49.030813 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:49.014385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58952 (local address 127.29.243.195:45017)
0809 04:59:49.015061 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:49.015075 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:49.015120 (+ 45us) server_negotiation.cc:407] Connection header received
0809 04:59:49.016780 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:49.016790 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:49.017229 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:49.017532 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:49.018891 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:49.020235 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:49.021496 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:49.022147 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:49.022291 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:49.024863 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:49.024873 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:49.024888 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:49.024932 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:49.027918 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:49.028557 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:49.028564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:49.028570 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:49.028733 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:49.029034 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:49.029037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:49.029039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:49.029560 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:49.029783 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:49.030139 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 04:59:49.030503 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":326,"threads_started":1}
I20260809 04:59:49.032280 22692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:49.114188 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 22490
2026-08-09T04:59:49Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25029 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:59:49 dist-test-slave-21b9 krb5kdc[22796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:59:49 dist-test-slave-21b9 krb5kdc[22796](info): set up 2 sockets
Aug 09 04:59:49 dist-test-slave-21b9 krb5kdc[22796](info): commencing operation
krb5kdc: starting...
W20260809 04:59:50.354496 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.015s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 04:59:50 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:59:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:59:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 04:59:50.638114 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44193
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:44193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:50.978894 22815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:50.979213 22815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:50.979364 22815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:50.986457 22815 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 04:59:50.986552 22815 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 04:59:50.986610 22815 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 04:59:50.986677 22815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:59:50.986750 22815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:50.986845 22815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:50.986907 22815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:59:50.986989 22815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:59:50.996763 22815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:44193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:50.998136 22815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:50.999505 22815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:51.011932 22822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:51.011965 22823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:51.011974 22826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:51.012974 22815 server_base.cc:1034] running on GCE node
I20260809 04:59:51.013778 22815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:51.015221 22815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:51.016490 22815 hybrid_clock.cc:648] HybridClock initialized: now 1786251591016429 us; error 56 us; skew 500 ppm
I20260809 04:59:51.016830 22815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:51 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251591, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:51.020020 22815 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 04:59:51.021513 22815 webserver.cc:460] Webserver started at http://127.29.243.254:36065/ using document root <none> and password file <none>
I20260809 04:59:51.022238 22815 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:51.022341 22815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:51.022656 22815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:51.025688 22815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "aad9dec6049048ffa753fc48d6502af8"
format_stamp: "Formatted at 2026-08-09 04:59:51 on dist-test-slave-21b9"
server_key: "0249b63ff1049ac62a59c5958552afd6"
server_key_iv: "0ff8aa20b2c3fa5b83707b7db021c943"
server_key_version: "encryptionkey@0"
I20260809 04:59:51.026438 22815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "aad9dec6049048ffa753fc48d6502af8"
format_stamp: "Formatted at 2026-08-09 04:59:51 on dist-test-slave-21b9"
server_key: "0249b63ff1049ac62a59c5958552afd6"
server_key_iv: "0ff8aa20b2c3fa5b83707b7db021c943"
server_key_version: "encryptionkey@0"
I20260809 04:59:51.031560 22815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 04:59:51.034929 22833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:51.036255 22815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:59:51.036425 22815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "aad9dec6049048ffa753fc48d6502af8"
format_stamp: "Formatted at 2026-08-09 04:59:51 on dist-test-slave-21b9"
server_key: "0249b63ff1049ac62a59c5958552afd6"
server_key_iv: "0ff8aa20b2c3fa5b83707b7db021c943"
server_key_version: "encryptionkey@0"
I20260809 04:59:51.036623 22815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:51.051494 22815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:51.055912 22815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:51.056176 22815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:51.074134 22815 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:44193
I20260809 04:59:51.074159 22922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:44193 every 8 connection(s)
I20260809 04:59:51.075613 22815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 04:59:51.079576 22924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:51.085649 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 22815
I20260809 04:59:51.085881 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 04:59:51.086556 30671 external_mini_cluster.cc:1428] Setting key 28639c15db2eb0ec0073efbfaf7885fc
I20260809 04:59:51.091355 22924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Bootstrap starting.
I20260809 04:59:51.095604 22924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:51.097034 22924 log.cc:824] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:51.100607 22924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: No bootstrap required, opened a new log
Aug 09 04:59:51 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:51.108958 22924 raft_consensus.cc:348] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 04:59:51.109408 22924 raft_consensus.cc:374] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:51.109519 22924 raft_consensus.cc:729] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad9dec6049048ffa753fc48d6502af8, State: Initialized, Role: FOLLOWER
I20260809 04:59:51.110184 22924 consensus_queue.cc:260] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 04:59:51.110464 22924 raft_consensus.cc:388] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:59:51.110626 22924 raft_consensus.cc:482] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:59:51.110812 22924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:51.112180 22929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:51.088811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.29.243.254:44193)
0809 04:59:51.090255 (+ 1444us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:51.090273 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:51.090332 (+ 59us) server_negotiation.cc:407] Connection header received
0809 04:59:51.093005 (+ 2673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:51.093040 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:51.093606 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:51.094107 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:51.096116 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:51.098012 (+ 1896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:51.099761 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:51.100422 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:51.100612 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:51.104593 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:51.104602 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:51.104621 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:51.104668 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:51.107905 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:51.108603 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:51.108610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:51.108617 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:51.108784 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:51.109172 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:51.109176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:51.109178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:51.109802 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:51.110124 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:51.110536 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 04:59:51.111007 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1180,"thread_start_us":923,"threads_started":1}
I20260809 04:59:51.113493 22924 raft_consensus.cc:504] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 04:59:51.114137 22924 leader_election.cc:302] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aad9dec6049048ffa753fc48d6502af8; no voters:
I20260809 04:59:51.114823 22924 leader_election.cc:288] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:59:51.114933 22931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:59:51.116097 22931 raft_consensus.cc:686] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 1 LEADER]: Becoming Leader. State: Replica: aad9dec6049048ffa753fc48d6502af8, State: Running, Role: LEADER
I20260809 04:59:51.116578 22931 consensus_queue.cc:237] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 04:59:51.117079 22924 sys_catalog.cc:564] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:59:51.119673 22935 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aad9dec6049048ffa753fc48d6502af8. Latest consensus state: current_term: 1 leader_uuid: "aad9dec6049048ffa753fc48d6502af8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } } }
I20260809 04:59:51.120421 22935 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:51.119462 22933 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aad9dec6049048ffa753fc48d6502af8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } } }
I20260809 04:59:51.121486 22933 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:59:51.123592 22947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:59:51.129236 22947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:59:51.137691 22947 catalog_manager.cc:1347] Generated new cluster ID: 12053e1ab25f498f9aa70e5ad201284c
I20260809 04:59:51.137830 22947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:59:51.170842 22947 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:59:51.172801 22947 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:59:51.182054 22947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Generated new TSK 0
I20260809 04:59:51.182893 22947 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 04:59:51.259894 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44193
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:51.609794 22970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:51.610092 22970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:51.610339 22970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:51.617741 22970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:51.617864 22970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:51.618021 22970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 04:59:51.628103 22970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:51.629460 22970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:51.630784 22970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:51.643945 22981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:51.643988 22987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:51.644021 22984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:51.644634 22970 server_base.cc:1034] running on GCE node
I20260809 04:59:51.645182 22970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:51.646044 22970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:51.647279 22970 hybrid_clock.cc:648] HybridClock initialized: now 1786251591647201 us; error 48 us; skew 500 ppm
I20260809 04:59:51.647625 22970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:51 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251591, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:51.650656 22970 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 04:59:51.652168 22970 webserver.cc:460] Webserver started at http://127.29.243.193:41307/ using document root <none> and password file <none>
I20260809 04:59:51.652807 22970 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:51.652907 22970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:51.653180 22970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:51.656078 22970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "e30cd81995b940da8add19b81be9081d"
format_stamp: "Formatted at 2026-08-09 04:59:51 on dist-test-slave-21b9"
server_key: "6448269b4d177f1c7c0ffaebc5a5c0af"
server_key_iv: "21a983c86167af4df0c1e0a8a7c45609"
server_key_version: "encryptionkey@0"
I20260809 04:59:51.656764 22970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "e30cd81995b940da8add19b81be9081d"
format_stamp: "Formatted at 2026-08-09 04:59:51 on dist-test-slave-21b9"
server_key: "6448269b4d177f1c7c0ffaebc5a5c0af"
server_key_iv: "21a983c86167af4df0c1e0a8a7c45609"
server_key_version: "encryptionkey@0"
I20260809 04:59:51.661902 22970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:59:51.665510 22996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:51.666844 22970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:59:51.667004 22970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "e30cd81995b940da8add19b81be9081d"
format_stamp: "Formatted at 2026-08-09 04:59:51 on dist-test-slave-21b9"
server_key: "6448269b4d177f1c7c0ffaebc5a5c0af"
server_key_iv: "21a983c86167af4df0c1e0a8a7c45609"
server_key_version: "encryptionkey@0"
I20260809 04:59:51.667192 22970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:51.696858 22970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:51.700812 22970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:51.701068 22970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:51.701982 22970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:51.703248 22970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:51.703342 22970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:51.703433 22970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:51.703496 22970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:51.733908 22970 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:45921
I20260809 04:59:51.733951 23172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:45921 every 8 connection(s)
I20260809 04:59:51.735224 22970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 04:59:51.739564 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 22970
I20260809 04:59:51.739801 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 04:59:51.740621 30671 external_mini_cluster.cc:1428] Setting key 4e620cb1673d55365625d0c1ef8fea85
Aug 09 04:59:51 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251591, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:51.756494 23177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:51.737789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:34311 (local address 127.29.243.254:44193)
0809 04:59:51.738231 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:51.738236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:51.739152 (+ 916us) server_negotiation.cc:407] Connection header received
0809 04:59:51.741623 (+ 2471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:51.741629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:51.741711 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:51.741815 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:51.743763 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:51.744483 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:51.746606 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:51.747004 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:51.747095 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:51.751031 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:51.751036 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:51.751039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:51.751071 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:51.753393 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:51.754187 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:51.754191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:51.754193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:51.754301 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:51.754716 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:51.754719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:51.754721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:51.754954 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:51.755077 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:51.755915 (+ 838us) server_negotiation.cc:299] Negotiation successful
0809 04:59:51.756145 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":222,"threads_started":1}
I20260809 04:59:51.757263 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:51.738249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:34311)
0809 04:59:51.738945 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:51.739012 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:51.741373 (+ 2361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:51.742102 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:51.742113 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:51.742723 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:51.743561 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:51.743588 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:51.744632 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:51.744637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:51.746470 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:51.746485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:51.747217 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:51.747224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:51.747526 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:51.748209 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:51.748238 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:51.750802 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:51.753538 (+ 2736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:51.753547 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:51.753562 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:51.754088 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:51.754388 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:51.754392 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:51.754394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:51.754628 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:51.755071 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:51.755078 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:51.755496 (+ 418us) client_negotiation.cc:769] Sending connection context
0809 04:59:51.755780 (+ 284us) client_negotiation.cc:240] Negotiation successful
0809 04:59:51.756178 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":280,"threads_started":1}
I20260809 04:59:51.758718 23174 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44193
I20260809 04:59:51.759014 23174 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:51.759759 23174 heartbeater.cc:511] Master 127.29.243.254:44193 requested a full tablet report, sending...
I20260809 04:59:51.762030 22861 ts_manager.cc:194] Registered new tserver with Master: e30cd81995b940da8add19b81be9081d (127.29.243.193:45921)
I20260809 04:59:51.763581 22861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:34311
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 04:59:51.802290 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44193
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:52.149001 23183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:52.149312 23183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:52.149461 23183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:52.156543 23183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:52.156661 23183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:52.156807 23183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 04:59:52.166460 23183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:52.167815 23183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:52.169304 23183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:52.182302 23192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:52.182271 23193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:52.182720 23196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:52.183056 23183 server_base.cc:1034] running on GCE node
I20260809 04:59:52.183688 23183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:52.184619 23183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:52.185818 23183 hybrid_clock.cc:648] HybridClock initialized: now 1786251592185763 us; error 39 us; skew 500 ppm
I20260809 04:59:52.186117 23183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:52 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:52.189155 23183 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 04:59:52.190627 23183 webserver.cc:460] Webserver started at http://127.29.243.194:39497/ using document root <none> and password file <none>
I20260809 04:59:52.191347 23183 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:52.191481 23183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:52.191778 23183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:52.194651 23183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "56e96fe9ef0a464e91582c6bfe6bb077"
format_stamp: "Formatted at 2026-08-09 04:59:52 on dist-test-slave-21b9"
server_key: "a87e684cd0e93444d8700bf34da36f80"
server_key_iv: "52a0e759d16fc31578b843027dae97b2"
server_key_version: "encryptionkey@0"
I20260809 04:59:52.195443 23183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "56e96fe9ef0a464e91582c6bfe6bb077"
format_stamp: "Formatted at 2026-08-09 04:59:52 on dist-test-slave-21b9"
server_key: "a87e684cd0e93444d8700bf34da36f80"
server_key_iv: "52a0e759d16fc31578b843027dae97b2"
server_key_version: "encryptionkey@0"
I20260809 04:59:52.200445 23183 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:59:52.203822 23206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:52.205128 23183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:59:52.205282 23183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "56e96fe9ef0a464e91582c6bfe6bb077"
format_stamp: "Formatted at 2026-08-09 04:59:52 on dist-test-slave-21b9"
server_key: "a87e684cd0e93444d8700bf34da36f80"
server_key_iv: "52a0e759d16fc31578b843027dae97b2"
server_key_version: "encryptionkey@0"
I20260809 04:59:52.205484 23183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:52.217392 23183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:52.221211 23183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:52.221473 23183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:52.222375 23183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:52.223714 23183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:52.223794 23183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:52.223910 23183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:52.223965 23183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:52.254705 23183 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:42213
I20260809 04:59:52.254737 23400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:42213 every 8 connection(s)
I20260809 04:59:52.256023 23183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 04:59:52.260339 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 23183
I20260809 04:59:52.260526 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 04:59:52.260970 30671 external_mini_cluster.cc:1428] Setting key 82544266fac31e6ef25a21d9678945aa
Aug 09 04:59:52 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:52.275652 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:52.258537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:34637 (local address 127.29.243.254:44193)
0809 04:59:52.258954 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:52.258959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:52.259862 (+ 903us) server_negotiation.cc:407] Connection header received
0809 04:59:52.261740 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:52.261746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:52.261828 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:52.261934 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:52.263434 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.264128 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.265616 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.266044 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.266153 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:52.269514 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:52.269519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:52.269523 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:52.269556 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:52.272343 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:52.273169 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:52.273173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:52.273175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:52.273300 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:52.273732 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:52.273735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:52.273737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:52.273991 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:52.274098 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:52.274990 (+ 892us) server_negotiation.cc:299] Negotiation successful
0809 04:59:52.275223 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":199,"threads_started":1}
I20260809 04:59:52.276401 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:52.258953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:34637)
0809 04:59:52.259699 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:52.259738 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:52.261473 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:52.262175 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:52.262183 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:52.262683 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:52.263238 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:52.263254 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.264268 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.264272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:52.265475 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:52.265492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.266271 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.266280 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:52.266495 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:52.267109 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:52.267132 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:52.269274 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:52.272490 (+ 3216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:52.272499 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:52.272515 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:52.273065 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:52.273393 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:52.273397 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:52.273400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:52.273641 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:52.274106 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:52.274114 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:52.274517 (+ 403us) client_negotiation.cc:769] Sending connection context
0809 04:59:52.274815 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 04:59:52.275225 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":298,"threads_started":1}
I20260809 04:59:52.277851 23401 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44193
I20260809 04:59:52.278182 23401 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:52.278910 23401 heartbeater.cc:511] Master 127.29.243.254:44193 requested a full tablet report, sending...
I20260809 04:59:52.280452 22861 ts_manager.cc:194] Registered new tserver with Master: 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213)
I20260809 04:59:52.281262 22861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:34637
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 04:59:52.324656 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44193
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:59:52.660140 23407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:59:52.660413 23407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:59:52.660506 23407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:59:52.667611 23407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:59:52.667724 23407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:59:52.667845 23407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 04:59:52.677526 23407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:59:52.678799 23407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:59:52.680114 23407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:59:52.693005 23416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:52.692939 23418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:59:52.692984 23420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:59:52.693820 23407 server_base.cc:1034] running on GCE node
I20260809 04:59:52.694362 23407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:59:52.695232 23407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:59:52.696444 23407 hybrid_clock.cc:648] HybridClock initialized: now 1786251592696386 us; error 52 us; skew 500 ppm
I20260809 04:59:52.696743 23407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:59:52 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:59:52.699747 23407 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 04:59:52.701072 23407 webserver.cc:460] Webserver started at http://127.29.243.195:33689/ using document root <none> and password file <none>
I20260809 04:59:52.701704 23407 fs_manager.cc:359] Metadata directory not provided
I20260809 04:59:52.701804 23407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:59:52.702057 23407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:59:52.704890 23407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "a16fb27521a24539a0fa4215bb03cd8b"
format_stamp: "Formatted at 2026-08-09 04:59:52 on dist-test-slave-21b9"
server_key: "af8d1b521cd247c7c4d4c0666dd7d7b4"
server_key_iv: "275dda783bd733f3bbc041bbb9a74d7c"
server_key_version: "encryptionkey@0"
I20260809 04:59:52.705597 23407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "a16fb27521a24539a0fa4215bb03cd8b"
format_stamp: "Formatted at 2026-08-09 04:59:52 on dist-test-slave-21b9"
server_key: "af8d1b521cd247c7c4d4c0666dd7d7b4"
server_key_iv: "275dda783bd733f3bbc041bbb9a74d7c"
server_key_version: "encryptionkey@0"
I20260809 04:59:52.710837 23407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:59:52.714422 23432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:52.715798 23407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 04:59:52.715955 23407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "a16fb27521a24539a0fa4215bb03cd8b"
format_stamp: "Formatted at 2026-08-09 04:59:52 on dist-test-slave-21b9"
server_key: "af8d1b521cd247c7c4d4c0666dd7d7b4"
server_key_iv: "275dda783bd733f3bbc041bbb9a74d7c"
server_key_version: "encryptionkey@0"
I20260809 04:59:52.716147 23407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:59:52.746061 23407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:59:52.749899 23407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:59:52.750171 23407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:59:52.751127 23407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:59:52.752516 23407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:59:52.752592 23407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:52.752722 23407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:59:52.752776 23407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:59:52.766614 23174 heartbeater.cc:503] Master 127.29.243.254:44193 was elected leader, sending a full tablet report...
I20260809 04:59:52.783253 23407 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:38481
I20260809 04:59:52.783344 23624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:38481 every 8 connection(s)
I20260809 04:59:52.784662 23407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 04:59:52.793044 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 23407
I20260809 04:59:52.793253 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 04:59:52.793723 30671 external_mini_cluster.cc:1428] Setting key 85a7317836f86dedeefeea4c47fdfd9e
Aug 09 04:59:52 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 04:59:52.805171 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:52.787220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:48167 (local address 127.29.243.254:44193)
0809 04:59:52.787757 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:52.787761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:52.788589 (+ 828us) server_negotiation.cc:407] Connection header received
0809 04:59:52.790428 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:52.790434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:52.790524 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:52.790621 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:52.792188 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.793144 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.794948 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.795507 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.795637 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:52.799508 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:52.799513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:52.799516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:52.799552 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:52.802194 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:52.802906 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:52.802910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:52.802912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:52.803032 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:52.803546 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:52.803550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:52.803552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:52.803834 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:52.803961 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:52.804716 (+ 755us) server_negotiation.cc:299] Negotiation successful
0809 04:59:52.804890 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":271,"threads_started":1}
I20260809 04:59:52.806185 23628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:52.787722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:48167)
0809 04:59:52.788415 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:52.788456 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:52.790157 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:52.790877 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:52.790885 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:52.791454 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:59:52.792019 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:52.792037 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.793340 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.793344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:52.794792 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:52.794806 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.795723 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.795730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:52.796027 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:52.796788 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:59:52.796830 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:59:52.799233 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:59:52.802340 (+ 3107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:52.802347 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:52.802362 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:52.802805 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:52.803143 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:59:52.803147 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:59:52.803149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:59:52.803400 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:59:52.803988 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:59:52.804009 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:59:52.804353 (+ 344us) client_negotiation.cc:769] Sending connection context
0809 04:59:52.804602 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 04:59:52.805023 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":270,"threads_started":1}
I20260809 04:59:52.807873 23625 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44193
I20260809 04:59:52.808238 23625 heartbeater.cc:464] Registering TS with master...
I20260809 04:59:52.809131 23625 heartbeater.cc:511] Master 127.29.243.254:44193 requested a full tablet report, sending...
I20260809 04:59:52.810660 22861 ts_manager.cc:194] Registered new tserver with Master: a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481)
I20260809 04:59:52.811540 22861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:48167
I20260809 04:59:52.821811 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:59:52.838990 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:52.826425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.29.243.254:44193)
0809 04:59:52.826615 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:52.826622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:52.827006 (+ 384us) server_negotiation.cc:407] Connection header received
0809 04:59:52.827171 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:52.827177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:52.827254 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:52.827364 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:52.828889 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.829617 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.830998 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.831422 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.831535 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:52.833885 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:52.833890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:52.833893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:52.833925 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:52.836407 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:52.837075 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:52.837079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:52.837081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:52.837198 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:52.837558 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:52.837563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:52.837566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:52.837872 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:52.838084 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:52.838462 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 04:59:52.838678 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:59:52.842971 22861 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:59:52.849762 22861 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:59:52.878870 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:52.862011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37248 (local address 127.29.243.194:42213)
0809 04:59:52.870213 (+ 8202us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:52.870221 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:52.870249 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:59:52.870399 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:52.870407 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:52.870633 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:52.871681 (+ 1048us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:52.871961 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.873516 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.876406 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.877601 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.878139 (+ 538us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:52.878302 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:52.878383 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 04:59:52.878549 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8051,"thread_start_us":7951,"threads_started":1}
I20260809 04:59:52.879089 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:52.861609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45921 (local address 127.0.0.1:44826)
0809 04:59:52.868559 (+ 6950us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:52.868611 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:52.868797 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:52.870026 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:52.870034 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:52.870066 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:52.870490 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:52.870505 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.872912 (+ 2407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.872918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:52.875475 (+ 2557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:52.875492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.877782 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.877789 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:52.878077 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:52.878201 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 04:59:52.878329 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 04:59:52.878588 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6653,"mutex_wait_us":99,"thread_start_us":7726,"threads_started":1}
I20260809 04:59:52.879114 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:52.861819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:42213 (local address 127.0.0.1:37248)
0809 04:59:52.869606 (+ 7787us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:52.869624 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:52.869735 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:52.870875 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:52.870880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:52.870908 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:52.871383 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:52.871396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.873682 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.873689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:52.876201 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:52.876218 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.877837 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.877842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:52.878130 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:52.878195 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 04:59:52.878309 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 04:59:52.878600 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7660,"mutex_wait_us":215,"thread_start_us":6276,"threads_started":1}
I20260809 04:59:52.879818 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:52.862160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44826 (local address 127.29.243.193:45921)
0809 04:59:52.869342 (+ 7182us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:52.869352 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:52.869382 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:59:52.869482 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:52.869489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:52.869766 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:52.870704 (+ 938us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:52.870963 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.872737 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.876099 (+ 3362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.877533 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.877720 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:52.877908 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:52.879353 (+ 1445us) server_negotiation.cc:299] Negotiation successful
0809 04:59:52.879521 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6983,"thread_start_us":6813,"threads_started":1}
I20260809 04:59:52.883210 23290 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0014e2808741b584cd81ea107711d2 (DEFAULT_TABLE table=test-table [id=89abe4fdd9f341e7b5c80e2a132add1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:52.883221 23065 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0014e2808741b584cd81ea107711d2 (DEFAULT_TABLE table=test-table [id=89abe4fdd9f341e7b5c80e2a132add1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:52.884684 23290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0014e2808741b584cd81ea107711d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:52.884723 23065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0014e2808741b584cd81ea107711d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:52.887475 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:52.870082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37450 (local address 127.29.243.195:38481)
0809 04:59:52.876092 (+ 6010us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:52.876100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:52.876646 (+ 546us) server_negotiation.cc:407] Connection header received
0809 04:59:52.876944 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:52.876953 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:52.877171 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:52.877328 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:52.878344 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.879831 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.884936 (+ 5105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.886302 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.886499 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:52.886670 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:52.886897 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 04:59:52.887078 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5841,"thread_start_us":5672,"threads_started":1}
I20260809 04:59:52.888537 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:52.870301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:38481 (local address 127.0.0.1:37450)
0809 04:59:52.876543 (+ 6242us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:52.876560 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:52.876748 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:52.877622 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:52.877627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:52.877651 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:52.878135 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:52.878149 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.880161 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.880166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:52.884710 (+ 4544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:52.884727 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:52.886511 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:52.886518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:52.886679 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:52.886711 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:59:52.887988 (+ 1277us) client_negotiation.cc:240] Negotiation successful
0809 04:59:52.888136 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6127,"mutex_wait_us":4,"thread_start_us":5961,"threads_started":1}
I20260809 04:59:52.892597 23513 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0014e2808741b584cd81ea107711d2 (DEFAULT_TABLE table=test-table [id=89abe4fdd9f341e7b5c80e2a132add1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:59:52.894330 23513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0014e2808741b584cd81ea107711d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:59:52.896682 23657 tablet_bootstrap.cc:492] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: Bootstrap starting.
I20260809 04:59:52.900007 23657 tablet_bootstrap.cc:654] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:52.900980 23657 log.cc:824] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:52.904080 23658 tablet_bootstrap.cc:492] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: Bootstrap starting.
I20260809 04:59:52.907421 23658 tablet_bootstrap.cc:654] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:52.907539 23657 tablet_bootstrap.cc:492] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: No bootstrap required, opened a new log
I20260809 04:59:52.907828 23657 ts_tablet_manager.cc:1397] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20260809 04:59:52.908679 23658 log.cc:824] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:52.908887 23662 tablet_bootstrap.cc:492] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: Bootstrap starting.
I20260809 04:59:52.912223 23658 tablet_bootstrap.cc:492] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: No bootstrap required, opened a new log
I20260809 04:59:52.912235 23662 tablet_bootstrap.cc:654] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: Neither blocks nor log segments found. Creating new log.
I20260809 04:59:52.912518 23658 ts_tablet_manager.cc:1397] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260809 04:59:52.913425 23662 log.cc:824] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:59:52.916397 23662 tablet_bootstrap.cc:492] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: No bootstrap required, opened a new log
I20260809 04:59:52.916285 23657 raft_consensus.cc:348] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 04:59:52.916662 23662 ts_tablet_manager.cc:1397] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260809 04:59:52.916790 23657 raft_consensus.cc:374] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:52.916905 23657 raft_consensus.cc:729] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e96fe9ef0a464e91582c6bfe6bb077, State: Initialized, Role: FOLLOWER
I20260809 04:59:52.917582 23657 consensus_queue.cc:260] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 04:59:52.918738 23401 heartbeater.cc:503] Master 127.29.243.254:44193 was elected leader, sending a full tablet report...
I20260809 04:59:52.920917 23657 ts_tablet_manager.cc:1428] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260809 04:59:52.921216 23658 raft_consensus.cc:348] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 04:59:52.921653 23658 raft_consensus.cc:374] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:52.921761 23658 raft_consensus.cc:729] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30cd81995b940da8add19b81be9081d, State: Initialized, Role: FOLLOWER
I20260809 04:59:52.922415 23658 consensus_queue.cc:260] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 04:59:52.924197 23658 ts_tablet_manager.cc:1428] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260809 04:59:52.926493 23662 raft_consensus.cc:348] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 04:59:52.926865 23662 raft_consensus.cc:374] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:59:52.926977 23662 raft_consensus.cc:729] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fb27521a24539a0fa4215bb03cd8b, State: Initialized, Role: FOLLOWER
I20260809 04:59:52.927549 23662 consensus_queue.cc:260] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 04:59:52.928532 23625 heartbeater.cc:503] Master 127.29.243.254:44193 was elected leader, sending a full tablet report...
I20260809 04:59:52.929194 23662 ts_tablet_manager.cc:1428] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: Time spent starting tablet: real 0.012s user 0.009s sys 0.002s
W20260809 04:59:52.987650 23176 tablet.cc:2369] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:59:53.008145 23402 tablet.cc:2369] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:59:53.030911 23671 raft_consensus.cc:482] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:59:53.031239 23671 raft_consensus.cc:504] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 04:59:53.033059 23671 leader_election.cc:288] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213), a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481)
W20260809 04:59:53.036335 23626 tablet.cc:2369] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:59:53.041580 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:53.033634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:32991 (local address 127.29.243.195:38481)
0809 04:59:53.033861 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:53.033867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:53.034693 (+ 826us) server_negotiation.cc:407] Connection header received
0809 04:59:53.034898 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:53.034902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:53.034966 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:53.035043 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:53.035703 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.036731 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.039322 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.040532 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.040703 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:53.040777 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:53.041189 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 04:59:53.041325 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 04:59:53.041720 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:53.033511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:52285 (local address 127.29.243.194:42213)
0809 04:59:53.033751 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:53.033757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:53.034744 (+ 987us) server_negotiation.cc:407] Connection header received
0809 04:59:53.034911 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:53.034917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:53.034994 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:53.035070 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:59:53.035783 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.036522 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.039327 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.040567 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.040702 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:53.040777 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:53.041338 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 04:59:53.041486 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:59:53.041730 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:53.033668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:38481 (local address 127.29.243.193:32991)
0809 04:59:53.034578 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:53.034594 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:53.034747 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:53.035071 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:53.035076 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:53.035101 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:53.035554 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:53.035570 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.036872 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.036876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:53.039129 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:53.039138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.040795 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.040802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:53.041022 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:53.041053 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 04:59:53.041130 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 04:59:53.041249 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":68,"thread_start_us":495,"threads_started":1}
I20260809 04:59:53.041817 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:53.033708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:42213 (local address 127.29.243.193:52285)
0809 04:59:53.034625 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0809 04:59:53.034644 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:59:53.034772 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:59:53.035104 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:59:53.035108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:59:53.035133 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:59:53.035619 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:53.035630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.036683 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.036687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:53.039120 (+ 2433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:59:53.039136 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.040948 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.040953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:59:53.041115 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:53.041148 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 04:59:53.041284 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 04:59:53.041421 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":167,"thread_start_us":475,"threads_started":1}
I20260809 04:59:53.042923 23324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0014e2808741b584cd81ea107711d2" candidate_uuid: "e30cd81995b940da8add19b81be9081d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" is_pre_election: true
I20260809 04:59:53.043155 23556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0014e2808741b584cd81ea107711d2" candidate_uuid: "e30cd81995b940da8add19b81be9081d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fb27521a24539a0fa4215bb03cd8b" is_pre_election: true
I20260809 04:59:53.043502 23324 raft_consensus.cc:2454] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e30cd81995b940da8add19b81be9081d in term 0.
I20260809 04:59:53.043661 23556 raft_consensus.cc:2454] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e30cd81995b940da8add19b81be9081d in term 0.
I20260809 04:59:53.044165 22998 leader_election.cc:302] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56e96fe9ef0a464e91582c6bfe6bb077, e30cd81995b940da8add19b81be9081d; no voters:
I20260809 04:59:53.044628 23671 raft_consensus.cc:2781] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:59:53.044755 23671 raft_consensus.cc:482] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:59:53.044848 23671 raft_consensus.cc:3037] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:53.047030 23671 raft_consensus.cc:504] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 04:59:53.047679 23671 leader_election.cc:288] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 1 election: Requested vote from peers 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213), a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481)
I20260809 04:59:53.048190 23556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0014e2808741b584cd81ea107711d2" candidate_uuid: "e30cd81995b940da8add19b81be9081d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fb27521a24539a0fa4215bb03cd8b"
I20260809 04:59:53.048190 23324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0014e2808741b584cd81ea107711d2" candidate_uuid: "e30cd81995b940da8add19b81be9081d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e96fe9ef0a464e91582c6bfe6bb077"
I20260809 04:59:53.048405 23556 raft_consensus.cc:3037] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:53.048405 23324 raft_consensus.cc:3037] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:59:53.051591 23556 raft_consensus.cc:2454] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e30cd81995b940da8add19b81be9081d in term 1.
I20260809 04:59:53.051591 23324 raft_consensus.cc:2454] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e30cd81995b940da8add19b81be9081d in term 1.
I20260809 04:59:53.052081 22998 leader_election.cc:302] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 56e96fe9ef0a464e91582c6bfe6bb077, a16fb27521a24539a0fa4215bb03cd8b, e30cd81995b940da8add19b81be9081d; no voters:
I20260809 04:59:53.052418 23671 raft_consensus.cc:2781] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:59:53.052884 23671 raft_consensus.cc:686] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 1 LEADER]: Becoming Leader. State: Replica: e30cd81995b940da8add19b81be9081d, State: Running, Role: LEADER
I20260809 04:59:53.053432 23671 consensus_queue.cc:237] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 04:59:53.058012 22858 catalog_manager.cc:5495] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d reported cstate change: term changed from 0 to 1, leader changed from <none> to e30cd81995b940da8add19b81be9081d (127.29.243.193). New cstate: current_term: 1 leader_uuid: "e30cd81995b940da8add19b81be9081d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } health_report { overall_health: UNKNOWN } } }
I20260809 04:59:53.077993 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:53.072661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44828 (local address 127.29.243.193:45921)
0809 04:59:53.072882 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:53.072886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:53.072905 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:59:53.073102 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:53.073105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:53.073194 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:53.073314 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:59:53.073672 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.074372 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.075844 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.076274 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.076385 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:53.076889 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:59:53.077410 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:59:53.077556 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:53.077656 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 04:59:53.077740 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:59:53.087127 23324 raft_consensus.cc:1261] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 1 FOLLOWER]: Refusing update from remote peer e30cd81995b940da8add19b81be9081d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:59:53.087127 23556 raft_consensus.cc:1261] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b [term 1 FOLLOWER]: Refusing update from remote peer e30cd81995b940da8add19b81be9081d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:59:53.088258 23671 consensus_queue.cc:1035] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:59:53.088567 23677 consensus_queue.cc:1035] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:59:53.101814 23681 mvcc.cc:205] Tried to move back new op lower bound from 7316486525272899584 to 7316486525150494720. Current Snapshot: MvccSnapshot[applied={T|T < 7316486525272899584}]
I20260809 04:59:53.103117 23680 mvcc.cc:205] Tried to move back new op lower bound from 7316486525272899584 to 7316486525150494720. Current Snapshot: MvccSnapshot[applied={T|T < 7316486525272899584 or (T in {7316486525272899584})}]
I20260809 04:59:53.105242 23682 mvcc.cc:205] Tried to move back new op lower bound from 7316486525272899584 to 7316486525150494720. Current Snapshot: MvccSnapshot[applied={T|T < 7316486525272899584}]
I20260809 04:59:53.116734 22860 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:59:53.117067 22860 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:59:53.120870 22860 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Sending DeleteTablet for 3 replicas of tablet 1e0014e2808741b584cd81ea107711d2
I20260809 04:59:53.122044 23065 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0014e2808741b584cd81ea107711d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:53 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:44826
I20260809 04:59:53.122031 23290 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0014e2808741b584cd81ea107711d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:53 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:37248
I20260809 04:59:53.122263 23513 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0014e2808741b584cd81ea107711d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:59:53 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:37450
I20260809 04:59:53.123401 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 22970
I20260809 04:59:53.134589 23695 tablet_replica.cc:330] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: stopping tablet replica
I20260809 04:59:53.135284 23695 raft_consensus.cc:2229] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:59:53.135993 23695 raft_consensus.cc:2258] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:53.138624 23695 ts_tablet_manager.cc:1905] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:53.139322 23696 tablet_replica.cc:330] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: stopping tablet replica
I20260809 04:59:53.141081 23696 raft_consensus.cc:2229] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:59:53.141965 23696 raft_consensus.cc:2258] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:53.142364 23698 tablet_replica.cc:330] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: stopping tablet replica
I20260809 04:59:53.142808 23698 raft_consensus.cc:2229] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:59:53.143488 23698 raft_consensus.cc:2258] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:59:53.145630 23696 ts_tablet_manager.cc:1905] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:53.145697 23698 ts_tablet_manager.cc:1905] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:59:53.149236 23695 ts_tablet_manager.cc:1918] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:59:53.149415 23695 log.cc:1196] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/1e0014e2808741b584cd81ea107711d2
I20260809 04:59:53.149955 23695 ts_tablet_manager.cc:1939] T 1e0014e2808741b584cd81ea107711d2 P 56e96fe9ef0a464e91582c6bfe6bb077: Deleting consensus metadata
I20260809 04:59:53.151883 22838 catalog_manager.cc:4841] TS 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213): tablet 1e0014e2808741b584cd81ea107711d2 (table test-table [id=89abe4fdd9f341e7b5c80e2a132add1c]) successfully deleted
I20260809 04:59:53.152745 23698 ts_tablet_manager.cc:1918] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 09 04:59:53 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 04:59:53.152937 23698 log.cc:1196] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/1e0014e2808741b584cd81ea107711d2
I20260809 04:59:53.153478 23698 ts_tablet_manager.cc:1939] T 1e0014e2808741b584cd81ea107711d2 P a16fb27521a24539a0fa4215bb03cd8b: Deleting consensus metadata
I20260809 04:59:53.155383 22839 catalog_manager.cc:4841] TS a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481): tablet 1e0014e2808741b584cd81ea107711d2 (table test-table [id=89abe4fdd9f341e7b5c80e2a132add1c]) successfully deleted
I20260809 04:59:53.157943 23696 ts_tablet_manager.cc:1918] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:59:53.158115 23696 log.cc:1196] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/1e0014e2808741b584cd81ea107711d2
I20260809 04:59:53.158687 23696 ts_tablet_manager.cc:1939] T 1e0014e2808741b584cd81ea107711d2 P e30cd81995b940da8add19b81be9081d: Deleting consensus metadata
I20260809 04:59:53.160341 22839 catalog_manager.cc:4841] TS e30cd81995b940da8add19b81be9081d (127.29.243.193:45921): tablet 1e0014e2808741b584cd81ea107711d2 (table test-table [id=89abe4fdd9f341e7b5c80e2a132add1c]) successfully deleted
I20260809 04:59:53.162246 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:53.124742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44832 (local address 127.29.243.193:45921)
0809 04:59:53.124942 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:53.124949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:53.135915 (+ 10966us) server_negotiation.cc:407] Connection header received
0809 04:59:53.136243 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:53.136250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:53.136372 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:53.136474 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:53.138953 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.140680 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.143016 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.144146 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.144272 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:53.154244 (+ 9972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:53.154258 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:53.154277 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:53.154326 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:53.158934 (+ 4608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:53.159953 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:53.159963 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:53.159966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:53.160156 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:53.160616 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:53.160621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:53.160624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:53.161007 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:53.161292 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:53.161673 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 04:59:53.161908 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 04:59:53.163121 23164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:53.265923 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 22970
I20260809 04:59:53.296815 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 23183
Aug 09 04:59:53 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 04:59:53.313020 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:53.297936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37262 (local address 127.29.243.194:42213)
0809 04:59:53.298127 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:53.298132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:53.298152 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:59:53.298289 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:53.298293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:53.298357 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:53.298470 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:53.299860 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.300602 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.302178 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.302580 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.302673 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:53.307212 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:53.307225 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:53.307241 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:53.307300 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:53.310184 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:53.310940 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:53.310949 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:53.310951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:53.311086 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:53.311515 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:53.311518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:53.311521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:53.311922 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:53.312164 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:53.312523 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 04:59:53.312694 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:59:53.313700 23392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:53.403220 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 23183
I20260809 04:59:53.428560 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 23407
Aug 09 04:59:53 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 04:59:53.443259 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:59:53.429352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.29.243.195:38481)
0809 04:59:53.429562 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 04:59:53.429568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:59:53.429591 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:59:53.429768 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:59:53.429773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:59:53.429853 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:59:53.429943 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:59:53.431213 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.431960 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.433339 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:59:53.433712 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:59:53.433801 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:59:53.438032 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:59:53.438049 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:59:53.438069 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:59:53.438103 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:59:53.440818 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:53.441468 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:53.441475 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:53.441477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:53.441637 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:59:53.441978 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:59:53.441981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:59:53.441983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:59:53.442304 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:59:53.442542 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 04:59:53.442779 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 04:59:53.442941 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:59:53.444001 23613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:53.532142 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 23407
I20260809 04:59:53.557801 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 22815
I20260809 04:59:53.558970 22918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:59:53.668725 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 22815
I20260809 05:00:13.698484 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44193
--webserver_interface=127.29.243.254
--webserver_port=36065
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:44193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:00:14.048622 23708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:14.048923 23708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:14.049021 23708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:14.056190 23708 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 05:00:14.056298 23708 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 05:00:14.056346 23708 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 05:00:14.056399 23708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:00:14.056452 23708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:14.056530 23708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:14.056579 23708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:00:14.056632 23708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:00:14.066625 23708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:44193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=36065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:14.067961 23708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:14.069355 23708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:14.093806 23720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:14.093761 23721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:14.093711 23724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:14.094336 23708 server_base.cc:1034] running on GCE node
I20260809 05:00:14.095429 23708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:14.097170 23708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:14.098455 23708 hybrid_clock.cc:648] HybridClock initialized: now 1786251614098380 us; error 32 us; skew 500 ppm
I20260809 05:00:14.098889 23708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:14 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251614, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:14.105576 23708 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:00:14.107411 23708 webserver.cc:460] Webserver started at http://127.29.243.254:36065/ using document root <none> and password file <none>
I20260809 05:00:14.108270 23708 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:14.108421 23708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:14.115218 23708 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.005s
I20260809 05:00:14.118541 23738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:14.120112 23708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:00:14.120298 23708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "aad9dec6049048ffa753fc48d6502af8"
format_stamp: "Formatted at 2026-08-09 04:59:51 on dist-test-slave-21b9"
server_key: "0249b63ff1049ac62a59c5958552afd6"
server_key_iv: "0ff8aa20b2c3fa5b83707b7db021c943"
server_key_version: "encryptionkey@0"
I20260809 05:00:14.121367 23708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:14.140513 23708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:14.144516 23708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:14.144788 23708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:14.162199 23708 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:44193
I20260809 05:00:14.162223 23815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:44193 every 8 connection(s)
I20260809 05:00:14.163813 23708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:00:14.170930 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 23708
I20260809 05:00:14.171597 23817 sys_catalog.cc:263] Verifying existing consensus state
I20260809 05:00:14.172053 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:45921
--local_ip_for_outbound_sockets=127.29.243.193
--tserver_master_addrs=127.29.243.254:44193
--webserver_port=41307
--webserver_interface=127.29.243.193
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:00:14.176294 23817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Bootstrap starting.
I20260809 05:00:14.198242 23817 log.cc:824] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Log is configured to *not* fsync() on all Append() calls
I20260809 05:00:14.210855 23817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 05:00:14.211480 23817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Bootstrap complete.
I20260809 05:00:14.220916 23817 raft_consensus.cc:348] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 05:00:14.221758 23817 raft_consensus.cc:729] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad9dec6049048ffa753fc48d6502af8, State: Initialized, Role: FOLLOWER
I20260809 05:00:14.222404 23817 consensus_queue.cc:260] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 05:00:14.222627 23817 raft_consensus.cc:388] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:00:14.222746 23817 raft_consensus.cc:482] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:00:14.222913 23817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 1 FOLLOWER]: Advancing to term 2
I20260809 05:00:14.227171 23817 raft_consensus.cc:504] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 05:00:14.227762 23817 leader_election.cc:302] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aad9dec6049048ffa753fc48d6502af8; no voters:
I20260809 05:00:14.228384 23817 leader_election.cc:288] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 05:00:14.228531 23822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 05:00:14.230749 23817 sys_catalog.cc:564] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:00:14.235473 23822 raft_consensus.cc:686] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 2 LEADER]: Becoming Leader. State: Replica: aad9dec6049048ffa753fc48d6502af8, State: Running, Role: LEADER
I20260809 05:00:14.236145 23822 consensus_queue.cc:237] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 05:00:14.239039 23825 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "aad9dec6049048ffa753fc48d6502af8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } } }
I20260809 05:00:14.239544 23825 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: This master's current role is: LEADER
I20260809 05:00:14.239501 23826 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aad9dec6049048ffa753fc48d6502af8. Latest consensus state: current_term: 2 leader_uuid: "aad9dec6049048ffa753fc48d6502af8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } } }
I20260809 05:00:14.239780 23826 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: This master's current role is: LEADER
I20260809 05:00:14.248908 23838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:00:14.257041 23838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:00:14.258598 23838 catalog_manager.cc:1259] Loaded cluster ID: 12053e1ab25f498f9aa70e5ad201284c
I20260809 05:00:14.258797 23838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:00:14.263293 23838 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:00:14.263723 23848 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 05:00:14.264876 23838 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Loaded TSK: 0
I20260809 05:00:14.283988 23838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Generated new TSK 1
I20260809 05:00:14.284391 23838 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 05:00:14.588867 23819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:14.589187 23819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:14.589325 23819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:14.596748 23819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:14.596868 23819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:14.597016 23819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 05:00:14.606937 23819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:45921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=41307
--tserver_master_addrs=127.29.243.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:14.608349 23819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:14.609735 23819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:14.623759 23857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:14.624302 23859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:14.624447 23856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:14.624539 23819 server_base.cc:1034] running on GCE node
I20260809 05:00:14.625310 23819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:14.626384 23819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:14.627590 23819 hybrid_clock.cc:648] HybridClock initialized: now 1786251614627529 us; error 53 us; skew 500 ppm
I20260809 05:00:14.627903 23819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:14 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251614, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:14.631223 23819 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 05:00:14.632932 23819 webserver.cc:460] Webserver started at http://127.29.243.193:41307/ using document root <none> and password file <none>
I20260809 05:00:14.633622 23819 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:14.633754 23819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:14.639482 23819 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 05:00:14.642217 23878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:14.643635 23819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:00:14.643790 23819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "e30cd81995b940da8add19b81be9081d"
format_stamp: "Formatted at 2026-08-09 04:59:51 on dist-test-slave-21b9"
server_key: "6448269b4d177f1c7c0ffaebc5a5c0af"
server_key_iv: "21a983c86167af4df0c1e0a8a7c45609"
server_key_version: "encryptionkey@0"
I20260809 05:00:14.644800 23819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:14.668903 23819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:14.673316 23819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:14.673591 23819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:14.674656 23819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:00:14.676190 23819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:00:14.676285 23819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:14.676397 23819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:00:14.676472 23819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:14.707791 23819 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:45921
I20260809 05:00:14.707894 24075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:45921 every 8 connection(s)
I20260809 05:00:14.709353 23819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 05:00:14.715998 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 23819
I20260809 05:00:14.716799 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:42213
--local_ip_for_outbound_sockets=127.29.243.194
--tserver_master_addrs=127.29.243.254:44193
--webserver_port=39497
--webserver_interface=127.29.243.194
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:00:14 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251614, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:00:14.735986 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:14.712308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:53863 (local address 127.29.243.254:44193)
0809 05:00:14.712859 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:14.712871 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:14.713566 (+ 695us) server_negotiation.cc:407] Connection header received
0809 05:00:14.715857 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:14.715879 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:14.716339 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:14.716794 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:14.718382 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:14.720171 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:14.721718 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:14.722616 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:14.722791 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:14.727884 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:14.727894 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:14.727914 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:14.727967 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:14.731495 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:14.732461 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:14.732469 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:14.732475 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:14.732637 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:14.733097 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:14.733099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:14.733101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:14.733587 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:14.733736 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:14.734637 (+ 901us) server_negotiation.cc:299] Negotiation successful
0809 05:00:14.734965 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":226,"threads_started":1}
I20260809 05:00:14.736223 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:14.712535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:53863)
0809 05:00:14.713396 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:14.713443 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:14.715216 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:14.716986 (+ 1770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:14.717013 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:14.717567 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:00:14.718120 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:14.718138 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:14.720333 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:14.720337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:14.721568 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:14.721583 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:14.722824 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:14.722829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:14.723029 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:14.723702 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:00:14.723731 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:00:14.727578 (+ 3847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:00:14.731652 (+ 4074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:14.731662 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:14.731678 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:14.732303 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:14.732745 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:14.732749 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:14.732752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:14.733001 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:14.733721 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:00:14.733729 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:00:14.734132 (+ 403us) client_negotiation.cc:769] Sending connection context
0809 05:00:14.734422 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 05:00:14.735003 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":337,"threads_started":1}
I20260809 05:00:14.739529 24076 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44193
I20260809 05:00:14.739843 24076 heartbeater.cc:464] Registering TS with master...
I20260809 05:00:14.740571 24076 heartbeater.cc:511] Master 127.29.243.254:44193 requested a full tablet report, sending...
I20260809 05:00:14.742882 23766 ts_manager.cc:194] Registered new tserver with Master: e30cd81995b940da8add19b81be9081d (127.29.243.193:45921)
I20260809 05:00:14.744499 23766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:53863
W20260809 05:00:15.067719 24083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:15.068059 24083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:15.068187 24083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:15.075570 24083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:15.075721 24083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:15.075870 24083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 05:00:15.085949 24083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:42213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=39497
--tserver_master_addrs=127.29.243.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:15.087486 24083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:15.088907 24083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:15.101905 24088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:15.101941 24096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:15.101974 24089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:15.102828 24083 server_base.cc:1034] running on GCE node
I20260809 05:00:15.103479 24083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:15.104408 24083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:15.105620 24083 hybrid_clock.cc:648] HybridClock initialized: now 1786251615105559 us; error 50 us; skew 500 ppm
I20260809 05:00:15.105935 24083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:15 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251615, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:15.108948 24083 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 05:00:15.110397 24083 webserver.cc:460] Webserver started at http://127.29.243.194:39497/ using document root <none> and password file <none>
I20260809 05:00:15.111088 24083 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:15.111218 24083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:15.116451 24083 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:00:15.119052 24111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:15.120405 24083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:00:15.120570 24083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "56e96fe9ef0a464e91582c6bfe6bb077"
format_stamp: "Formatted at 2026-08-09 04:59:52 on dist-test-slave-21b9"
server_key: "a87e684cd0e93444d8700bf34da36f80"
server_key_iv: "52a0e759d16fc31578b843027dae97b2"
server_key_version: "encryptionkey@0"
I20260809 05:00:15.121506 24083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:15.151813 24083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:15.156044 24083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:15.156306 24083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:15.157258 24083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:00:15.158565 24083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:00:15.158661 24083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:15.158762 24083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:00:15.158836 24083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:15.191416 24083 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:42213
I20260809 05:00:15.191474 24286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:42213 every 8 connection(s)
I20260809 05:00:15.192870 24083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 05:00:15.201556 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 24083
I20260809 05:00:15.202183 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:38481
--local_ip_for_outbound_sockets=127.29.243.195
--tserver_master_addrs=127.29.243.254:44193
--webserver_port=33689
--webserver_interface=127.29.243.195
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:00:15 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251615, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:00:15.212329 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.195392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:46379 (local address 127.29.243.254:44193)
0809 05:00:15.195579 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:15.195584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:15.196707 (+ 1123us) server_negotiation.cc:407] Connection header received
0809 05:00:15.198582 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:15.198588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:15.198683 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:15.198775 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:15.200341 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.201053 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.202650 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.203062 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.203187 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.206629 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:15.206634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:15.206638 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:15.206671 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:15.209239 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:15.209980 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:15.209984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:15.209986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:15.210113 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:15.210534 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:15.210538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:15.210540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:15.210848 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:15.210988 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:15.211723 (+ 735us) server_negotiation.cc:299] Negotiation successful
0809 05:00:15.211978 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 05:00:15.213311 24290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.195805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:46379)
0809 05:00:15.196535 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:15.196574 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:15.198297 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:15.199037 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:15.199045 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:15.199591 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:00:15.200169 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.200186 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.201201 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.201205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.202509 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.202522 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.203306 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.203315 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.203553 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.204190 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:00:15.204211 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:00:15.206401 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:00:15.209398 (+ 2997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:15.209407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:15.209423 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:15.209849 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:15.210222 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:15.210227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:15.210230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:15.210414 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:15.210993 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:00:15.211001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:00:15.211351 (+ 350us) client_negotiation.cc:769] Sending connection context
0809 05:00:15.211599 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 05:00:15.212140 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":278,"threads_started":1}
I20260809 05:00:15.214769 24288 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44193
I20260809 05:00:15.215122 24288 heartbeater.cc:464] Registering TS with master...
I20260809 05:00:15.216001 24288 heartbeater.cc:511] Master 127.29.243.254:44193 requested a full tablet report, sending...
I20260809 05:00:15.217664 23766 ts_manager.cc:194] Registered new tserver with Master: 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213)
I20260809 05:00:15.218698 23766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:46379
W20260809 05:00:15.550107 24292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:15.550387 24292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:15.550486 24292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:15.557647 24292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:15.557760 24292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:15.557883 24292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 05:00:15.567664 24292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:38481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=33689
--tserver_master_addrs=127.29.243.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:15.568959 24292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:15.570343 24292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:15.583232 24304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:15.583201 24301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:15.583359 24300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:15.584348 24292 server_base.cc:1034] running on GCE node
I20260809 05:00:15.584961 24292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:15.585870 24292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:15.587076 24292 hybrid_clock.cc:648] HybridClock initialized: now 1786251615587029 us; error 52 us; skew 500 ppm
I20260809 05:00:15.587428 24292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:15 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251615, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:15.590397 24292 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 05:00:15.591789 24292 webserver.cc:460] Webserver started at http://127.29.243.195:33689/ using document root <none> and password file <none>
I20260809 05:00:15.592453 24292 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:15.592585 24292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:15.597823 24292 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 05:00:15.600459 24319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:15.601778 24292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:00:15.601956 24292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "a16fb27521a24539a0fa4215bb03cd8b"
format_stamp: "Formatted at 2026-08-09 04:59:52 on dist-test-slave-21b9"
server_key: "af8d1b521cd247c7c4d4c0666dd7d7b4"
server_key_iv: "275dda783bd733f3bbc041bbb9a74d7c"
server_key_version: "encryptionkey@0"
I20260809 05:00:15.602908 24292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:15.615355 24292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:15.619513 24292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:15.619756 24292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:15.620698 24292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:00:15.622036 24292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:00:15.622140 24292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:15.622241 24292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:00:15.622310 24292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:15.651988 24292 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:38481
I20260809 05:00:15.652043 24503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:38481 every 8 connection(s)
I20260809 05:00:15.653399 24292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 05:00:15.655063 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 24292
Aug 09 05:00:15 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251615, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:00:15.670917 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.656040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36328 (local address 127.29.243.254:44193)
0809 05:00:15.656206 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:15.656211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:15.656882 (+ 671us) server_negotiation.cc:407] Connection header received
0809 05:00:15.657091 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:15.657097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:15.657198 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:15.657318 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:15.659400 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.660156 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.661528 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.662011 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.662129 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.664969 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:15.664975 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:15.664979 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:15.665016 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:15.668105 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:15.668893 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:15.668897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:15.668900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:15.669033 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:15.669450 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:15.669454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:15.669456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:15.669724 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:15.669985 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:15.670276 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 05:00:15.670549 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 05:00:15.675334 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.656040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:47259 (local address 127.29.243.254:44193)
0809 05:00:15.658762 (+ 2722us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:15.658768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:15.658786 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:00:15.660212 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:15.660217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:15.660292 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:15.660382 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:15.661933 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.662682 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.664329 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.664954 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.665044 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.669023 (+ 3979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:15.669028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:15.669031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:15.669075 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:15.672298 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:15.673199 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:15.673203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:15.673206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:15.673332 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:15.673857 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:15.673863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:15.673867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:15.674132 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:15.674239 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:15.674847 (+ 608us) server_negotiation.cc:299] Negotiation successful
0809 05:00:15.675011 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":177,"server-negotiator.queue_time_us":2599,"thread_start_us":2313,"threads_started":1}
I20260809 05:00:15.676334 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.656697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:47259)
0809 05:00:15.657986 (+ 1289us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:15.658054 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:15.659941 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:15.660642 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:15.660650 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:15.661149 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:00:15.661759 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.661779 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.662874 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.662880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.664182 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.664196 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.665209 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.665217 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.665552 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.666040 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:00:15.666070 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:00:15.668724 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:00:15.672436 (+ 3712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:15.672443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:15.672457 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:15.673058 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:15.673472 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:15.673477 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:15.673480 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:15.673736 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:15.674237 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:00:15.674244 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:00:15.674549 (+ 305us) client_negotiation.cc:769] Sending connection context
0809 05:00:15.674768 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 05:00:15.675160 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":807,"threads_started":1}
I20260809 05:00:15.677620 24505 heartbeater.cc:347] Connected to a master server at 127.29.243.254:44193
I20260809 05:00:15.677906 24505 heartbeater.cc:464] Registering TS with master...
I20260809 05:00:15.678615 24505 heartbeater.cc:511] Master 127.29.243.254:44193 requested a full tablet report, sending...
I20260809 05:00:15.680014 23766 ts_manager.cc:194] Registered new tserver with Master: a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481)
I20260809 05:00:15.680783 23766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:47259
I20260809 05:00:15.685822 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:00:15.692613 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.686675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36340 (local address 127.29.243.254:44193)
0809 05:00:15.686873 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:15.686880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:15.687360 (+ 480us) server_negotiation.cc:407] Connection header received
0809 05:00:15.687600 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:15.687604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:15.687738 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:15.687858 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:00:15.688324 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.689048 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.690664 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.691046 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.691151 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.691714 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:00:15.691982 (+ 268us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 05:00:15.692334 (+ 352us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36340: token expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 05:00:15.692713 24509 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36340: token expired
W20260809 05:00:15.693194 23633 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.243.254:44193)
I20260809 05:00:15.705021 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.693784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36350 (local address 127.29.243.254:44193)
0809 05:00:15.693933 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:15.693937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:15.693952 (+ 15us) server_negotiation.cc:407] Connection header received
0809 05:00:15.693989 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:15.694002 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:15.694054 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:15.694127 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:15.695520 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.696181 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.697523 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.697902 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.697999 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.700511 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:15.700516 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:15.700519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:15.700549 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:15.702766 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:15.703402 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:15.703405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:15.703407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:15.703526 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:15.703827 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:15.703830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:15.703832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:15.704116 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:15.704309 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:15.704575 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 05:00:15.704729 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 05:00:15.708768 23766 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:00:15.717731 23766 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:00:15.740788 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.729666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:42213 (local address 127.0.0.1:36648)
0809 05:00:15.732146 (+ 2480us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:15.732170 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:15.732304 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:15.733179 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:15.733187 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:15.733223 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:00:15.733642 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.733654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.735507 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.735513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.737810 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.737824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.739839 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.739845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.740069 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.740126 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 05:00:15.740256 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 05:00:15.740434 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2356,"thread_start_us":2214,"threads_started":1}
I20260809 05:00:15.740830 24524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.730244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36648 (local address 127.29.243.194:42213)
0809 05:00:15.731660 (+ 1416us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:15.731670 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:15.732277 (+ 607us) server_negotiation.cc:407] Connection header received
0809 05:00:15.732529 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:15.732539 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:15.732942 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:15.733150 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:00:15.733864 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.735336 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.737964 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.739649 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.739813 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.740010 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:15.740314 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 05:00:15.740486 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1233,"thread_start_us":1074,"threads_started":1}
I20260809 05:00:15.742306 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.731360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:38481 (local address 127.0.0.1:47258)
0809 05:00:15.734262 (+ 2902us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:15.734281 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:15.734384 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:15.735363 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:15.735367 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:15.735392 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:00:15.735767 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.735776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.737236 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.737241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.739541 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.739555 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.741504 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.741511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.741668 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.741701 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 05:00:15.741772 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 05:00:15.741892 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2788,"mutex_wait_us":24,"thread_start_us":2660,"threads_started":1}
I20260809 05:00:15.742731 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.729380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45921 (local address 127.0.0.1:46790)
0809 05:00:15.730220 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:15.730274 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:15.730447 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:15.734611 (+ 4164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:15.734615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:15.734641 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:00:15.734996 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.735009 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.736921 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.736926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.739561 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.739578 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.741896 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.741902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.742071 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.742103 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 05:00:15.742205 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 05:00:15.742336 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":22,"thread_start_us":380,"threads_started":1}
I20260809 05:00:15.742935 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.732060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47258 (local address 127.29.243.195:38481)
0809 05:00:15.732794 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:15.732803 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:15.734513 (+ 1710us) server_negotiation.cc:407] Connection header received
0809 05:00:15.734615 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:15.734622 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:15.735031 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:15.735193 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:00:15.735941 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.737094 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.739682 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.741320 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.741672 (+ 352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.741846 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:15.741932 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 05:00:15.742097 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":423,"threads_started":1}
I20260809 05:00:15.743517 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.729667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.29.243.193:45921)
0809 05:00:15.733839 (+ 4172us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:15.733846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:15.733873 (+ 27us) server_negotiation.cc:407] Connection header received
0809 05:00:15.733969 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:15.733976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:15.734193 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:15.734376 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:00:15.735176 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.736791 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.739770 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.741348 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.741572 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.741735 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:15.743014 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0809 05:00:15.743173 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4015,"thread_start_us":3879,"threads_started":1}
I20260809 05:00:15.745434 24190 tablet_service.cc:1467] Processing CreateTablet for tablet cb6af646723541959493821475fa3600 (DEFAULT_TABLE table=test-table [id=86e2a4efeaa3494c952d2e9b4bcd8fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:00:15.747023 24190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb6af646723541959493821475fa3600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:00:15.746609 24393 tablet_service.cc:1467] Processing CreateTablet for tablet cb6af646723541959493821475fa3600 (DEFAULT_TABLE table=test-table [id=86e2a4efeaa3494c952d2e9b4bcd8fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:00:15.747766 24076 heartbeater.cc:503] Master 127.29.243.254:44193 was elected leader, sending a full tablet report...
I20260809 05:00:15.748694 24393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb6af646723541959493821475fa3600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:00:15.749423 23960 tablet_service.cc:1467] Processing CreateTablet for tablet cb6af646723541959493821475fa3600 (DEFAULT_TABLE table=test-table [id=86e2a4efeaa3494c952d2e9b4bcd8fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:00:15.750554 23960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb6af646723541959493821475fa3600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:00:15.762295 24527 tablet_bootstrap.cc:492] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Bootstrap starting.
I20260809 05:00:15.765172 24533 tablet_bootstrap.cc:492] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: Bootstrap starting.
I20260809 05:00:15.766618 24527 tablet_bootstrap.cc:654] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Neither blocks nor log segments found. Creating new log.
I20260809 05:00:15.767961 24531 tablet_bootstrap.cc:492] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: Bootstrap starting.
I20260809 05:00:15.768003 24527 log.cc:824] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Log is configured to *not* fsync() on all Append() calls
I20260809 05:00:15.768644 24533 tablet_bootstrap.cc:654] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: Neither blocks nor log segments found. Creating new log.
I20260809 05:00:15.769953 24533 log.cc:824] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: Log is configured to *not* fsync() on all Append() calls
I20260809 05:00:15.770864 24531 tablet_bootstrap.cc:654] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: Neither blocks nor log segments found. Creating new log.
I20260809 05:00:15.771921 24531 log.cc:824] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: Log is configured to *not* fsync() on all Append() calls
I20260809 05:00:15.773312 24533 tablet_bootstrap.cc:492] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: No bootstrap required, opened a new log
I20260809 05:00:15.773675 24533 ts_tablet_manager.cc:1397] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260809 05:00:15.774567 24531 tablet_bootstrap.cc:492] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: No bootstrap required, opened a new log
I20260809 05:00:15.774801 24531 ts_tablet_manager.cc:1397] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260809 05:00:15.777493 24527 tablet_bootstrap.cc:492] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: No bootstrap required, opened a new log
I20260809 05:00:15.777812 24527 ts_tablet_manager.cc:1397] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20260809 05:00:15.782024 24531 raft_consensus.cc:348] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:15.782366 24531 raft_consensus.cc:374] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:00:15.782467 24531 raft_consensus.cc:729] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e96fe9ef0a464e91582c6bfe6bb077, State: Initialized, Role: FOLLOWER
I20260809 05:00:15.783069 24531 consensus_queue.cc:260] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:15.783888 24533 raft_consensus.cc:348] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:15.784355 24533 raft_consensus.cc:374] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:00:15.784484 24533 raft_consensus.cc:729] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fb27521a24539a0fa4215bb03cd8b, State: Initialized, Role: FOLLOWER
I20260809 05:00:15.784750 24288 heartbeater.cc:503] Master 127.29.243.254:44193 was elected leader, sending a full tablet report...
I20260809 05:00:15.785403 24531 ts_tablet_manager.cc:1428] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: Time spent starting tablet: real 0.011s user 0.005s sys 0.006s
I20260809 05:00:15.785238 24533 consensus_queue.cc:260] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:15.786679 24505 heartbeater.cc:503] Master 127.29.243.254:44193 was elected leader, sending a full tablet report...
I20260809 05:00:15.786846 24533 ts_tablet_manager.cc:1428] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260809 05:00:15.787674 24527 raft_consensus.cc:348] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:15.788079 24527 raft_consensus.cc:374] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:00:15.788193 24527 raft_consensus.cc:729] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30cd81995b940da8add19b81be9081d, State: Initialized, Role: FOLLOWER
I20260809 05:00:15.788784 24527 consensus_queue.cc:260] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:15.790467 24527 ts_tablet_manager.cc:1428] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
W20260809 05:00:15.905000 24507 tablet.cc:2369] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:00:15.943890 24540 raft_consensus.cc:482] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:00:15.944204 24540 raft_consensus.cc:504] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
W20260809 05:00:15.944875 24289 tablet.cc:2369] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:00:15.946086 24540 leader_election.cc:288] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e30cd81995b940da8add19b81be9081d (127.29.243.193:45921), 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213)
I20260809 05:00:15.951848 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.946537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:34661 (local address 127.29.243.193:45921)
0809 05:00:15.946754 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:15.946760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:15.946863 (+ 103us) server_negotiation.cc:407] Connection header received
0809 05:00:15.947062 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:15.947066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:15.947128 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:15.947204 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:00:15.947621 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.948401 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.950197 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.950956 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.951103 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.951179 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:15.951468 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 05:00:15.951597 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 05:00:15.951895 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.946396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45921 (local address 127.29.243.195:34661)
0809 05:00:15.946773 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:15.946795 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:15.946930 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:15.947244 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:15.947248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:15.947302 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:00:15.947483 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.947492 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.948547 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.948553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.950044 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.950058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.951104 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.951110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.951308 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.951345 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:00:15.951429 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 05:00:15.951592 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":3}
I20260809 05:00:15.952409 24524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.946609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:36347 (local address 127.29.243.194:42213)
0809 05:00:15.946818 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:15.946823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:15.947472 (+ 649us) server_negotiation.cc:407] Connection header received
0809 05:00:15.947649 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:15.947653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:15.947712 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:15.947784 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:00:15.948320 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.949008 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.950747 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.951568 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.951704 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.951776 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:15.952100 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 05:00:15.952229 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 05:00:15.952955 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:15.946595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:42213 (local address 127.29.243.195:36347)
0809 05:00:15.947371 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:15.947386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:15.947518 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:15.947809 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:15.947813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:15.947837 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:00:15.948169 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.948177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.949158 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.949163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.950592 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:15.950605 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:15.951764 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:15.951770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:15.951916 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:15.951947 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 05:00:15.952576 (+ 629us) client_negotiation.cc:240] Negotiation successful
0809 05:00:15.952682 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":20,"thread_start_us":410,"threads_started":1}
I20260809 05:00:15.952869 24001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb6af646723541959493821475fa3600" candidate_uuid: "a16fb27521a24539a0fa4215bb03cd8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30cd81995b940da8add19b81be9081d" is_pre_election: true
I20260809 05:00:15.953428 24001 raft_consensus.cc:2454] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a16fb27521a24539a0fa4215bb03cd8b in term 0.
I20260809 05:00:15.953953 24223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb6af646723541959493821475fa3600" candidate_uuid: "a16fb27521a24539a0fa4215bb03cd8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" is_pre_election: true
I20260809 05:00:15.954191 24325 leader_election.cc:302] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fb27521a24539a0fa4215bb03cd8b, e30cd81995b940da8add19b81be9081d; no voters:
I20260809 05:00:15.954541 24223 raft_consensus.cc:2454] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a16fb27521a24539a0fa4215bb03cd8b in term 0.
I20260809 05:00:15.954612 24540 raft_consensus.cc:2781] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:00:15.954728 24540 raft_consensus.cc:482] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:00:15.954825 24540 raft_consensus.cc:3037] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:00:15.957072 24540 raft_consensus.cc:504] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:15.957675 24540 leader_election.cc:288] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [CANDIDATE]: Term 1 election: Requested vote from peers e30cd81995b940da8add19b81be9081d (127.29.243.193:45921), 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213)
I20260809 05:00:15.958122 24001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb6af646723541959493821475fa3600" candidate_uuid: "a16fb27521a24539a0fa4215bb03cd8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30cd81995b940da8add19b81be9081d"
I20260809 05:00:15.958145 24223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb6af646723541959493821475fa3600" candidate_uuid: "a16fb27521a24539a0fa4215bb03cd8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e96fe9ef0a464e91582c6bfe6bb077"
I20260809 05:00:15.958307 24001 raft_consensus.cc:3037] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:00:15.958323 24223 raft_consensus.cc:3037] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:00:15.960569 24223 raft_consensus.cc:2454] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fb27521a24539a0fa4215bb03cd8b in term 1.
I20260809 05:00:15.960579 24001 raft_consensus.cc:2454] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fb27521a24539a0fa4215bb03cd8b in term 1.
I20260809 05:00:15.960953 24324 leader_election.cc:302] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56e96fe9ef0a464e91582c6bfe6bb077, a16fb27521a24539a0fa4215bb03cd8b; no voters:
I20260809 05:00:15.961263 24540 raft_consensus.cc:2781] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 1 FOLLOWER]: Leader election won for term 1
W20260809 05:00:15.961675 24077 tablet.cc:2369] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:00:15.961745 24540 raft_consensus.cc:686] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 1 LEADER]: Becoming Leader. State: Replica: a16fb27521a24539a0fa4215bb03cd8b, State: Running, Role: LEADER
I20260809 05:00:15.962293 24540 consensus_queue.cc:237] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:15.966765 23764 catalog_manager.cc:5495] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b reported cstate change: term changed from 0 to 1, leader changed from <none> to a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195). New cstate: current_term: 1 leader_uuid: "a16fb27521a24539a0fa4215bb03cd8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } health_report { overall_health: HEALTHY } } }
I20260809 05:00:16.038841 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:16.033551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47264 (local address 127.29.243.195:38481)
0809 05:00:16.033722 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:16.033726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:16.033803 (+ 77us) server_negotiation.cc:407] Connection header received
0809 05:00:16.033991 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:16.033997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:16.034079 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:16.034207 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:00:16.034611 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:16.035346 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:16.036850 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:16.037263 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:16.037349 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:16.037877 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:00:16.038377 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:00:16.038507 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:16.038569 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 05:00:16.038635 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 05:00:16.047137 24223 raft_consensus.cc:1261] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 1 FOLLOWER]: Refusing update from remote peer a16fb27521a24539a0fa4215bb03cd8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:00:16.047137 24001 raft_consensus.cc:1261] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Refusing update from remote peer a16fb27521a24539a0fa4215bb03cd8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:00:16.048153 24550 consensus_queue.cc:1035] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:00:16.048592 24540 consensus_queue.cc:1035] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:00:16.062647 24556 mvcc.cc:205] Tried to move back new op lower bound from 7316486619319177216 to 7316486618985312256. Current Snapshot: MvccSnapshot[applied={T|T < 7316486619319177216}]
I20260809 05:00:16.064320 24555 mvcc.cc:205] Tried to move back new op lower bound from 7316486619319177216 to 7316486618985312256. Current Snapshot: MvccSnapshot[applied={T|T < 7316486619319177216}]
I20260809 05:00:16.074431 23766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:00:16.074719 23766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:00:16.078137 23766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Sending DeleteTablet for 3 replicas of tablet cb6af646723541959493821475fa3600
I20260809 05:00:16.079617 23960 tablet_service.cc:1514] Processing DeleteTablet for tablet cb6af646723541959493821475fa3600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:00:16 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:46790
I20260809 05:00:16.079818 24190 tablet_service.cc:1514] Processing DeleteTablet for tablet cb6af646723541959493821475fa3600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:00:16 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:36648
I20260809 05:00:16.079947 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 23819
I20260809 05:00:16.079425 24393 tablet_service.cc:1514] Processing DeleteTablet for tablet cb6af646723541959493821475fa3600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:00:16 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:47258
I20260809 05:00:16.102591 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:16.081555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46798 (local address 127.29.243.193:45921)
0809 05:00:16.081741 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:16.081747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:16.081766 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:00:16.081826 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:16.081831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:16.081912 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:16.082019 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:16.084320 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:16.085625 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:16.087797 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:16.088522 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:16.088643 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:16.092593 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:16.092611 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:16.092636 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:16.092689 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:16.097003 (+ 4314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:16.098531 (+ 1528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:16.098543 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:16.098548 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:16.098754 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:16.099309 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:16.099314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:16.099318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:16.100180 (+ 862us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:16.100522 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:16.101851 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0809 05:00:16.102127 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 05:00:16.103739 24071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:00:16.117867 24568 tablet_replica.cc:330] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: stopping tablet replica
I20260809 05:00:16.117715 24570 tablet_replica.cc:330] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: stopping tablet replica
I20260809 05:00:16.118428 24568 raft_consensus.cc:2229] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:00:16.119073 24568 raft_consensus.cc:2258] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:00:16.119297 24570 raft_consensus.cc:2229] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:00:16.120008 24570 raft_consensus.cc:2258] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:00:16.121318 24568 ts_tablet_manager.cc:1905] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:00:16.122251 24570 ts_tablet_manager.cc:1905] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:00:16.128433 24568 ts_tablet_manager.cc:1918] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:00:16.128702 24568 log.cc:1196] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/cb6af646723541959493821475fa3600
I20260809 05:00:16.129421 24568 ts_tablet_manager.cc:1939] T cb6af646723541959493821475fa3600 P 56e96fe9ef0a464e91582c6bfe6bb077: Deleting consensus metadata
I20260809 05:00:16.129890 24570 ts_tablet_manager.cc:1918] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:00:16.130160 24570 log.cc:1196] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/cb6af646723541959493821475fa3600
I20260809 05:00:16.130817 24570 ts_tablet_manager.cc:1939] T cb6af646723541959493821475fa3600 P a16fb27521a24539a0fa4215bb03cd8b: Deleting consensus metadata
I20260809 05:00:16.131618 23740 catalog_manager.cc:4841] TS 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213): tablet cb6af646723541959493821475fa3600 (table test-table [id=86e2a4efeaa3494c952d2e9b4bcd8fb7]) successfully deleted
I20260809 05:00:16.132807 23743 catalog_manager.cc:4841] TS a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481): tablet cb6af646723541959493821475fa3600 (table test-table [id=86e2a4efeaa3494c952d2e9b4bcd8fb7]) successfully deleted
I20260809 05:00:16.268680 24567 tablet_replica.cc:330] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: stopping tablet replica
I20260809 05:00:16.268747 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 23819
W20260809 05:00:16.296973 23743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.243.193:45921 (error 108)
I20260809 05:00:16.297169 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 24083
I20260809 05:00:16.299579 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:16.298853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45921 (local address 127.0.0.1:46810)
0809 05:00:16.299306 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:16.299415 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":222,"threads_started":1}
W20260809 05:00:16.299883 23743 catalog_manager.cc:4584] TS e30cd81995b940da8add19b81be9081d (127.29.243.193:45921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb6af646723541959493821475fa3600: Network error: Client connection negotiation failed: client connection to 127.29.243.193:45921: connect: Connection refused (error 111)
I20260809 05:00:16.312718 24524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:16.298038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.29.243.194:42213)
0809 05:00:16.298234 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:16.298240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:16.298299 (+ 59us) server_negotiation.cc:407] Connection header received
0809 05:00:16.298517 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:16.298522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:16.298608 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:16.298717 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:16.300537 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:16.301235 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:16.303096 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:16.303567 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:16.303667 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:16.307031 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:16.307049 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:16.307066 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:16.307105 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:16.309932 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:16.310748 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:16.310755 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:16.310757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:16.310882 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:16.311304 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:16.311308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:16.311312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:16.311688 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:16.311934 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:16.312277 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 05:00:16.312443 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:00:16.313565 24279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:00:16.353734 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:16.353175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45921 (local address 127.0.0.1:46814)
0809 05:00:16.353429 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:16.353557 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 05:00:16.355016 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:16.354580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45921 (local address 127.0.0.1:46822)
0809 05:00:16.354739 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:16.354832 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 05:00:16.404736 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 24083
I20260809 05:00:16.415444 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:16.414956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45921 (local address 127.0.0.1:46838)
0809 05:00:16.415166 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:16.415256 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 05:00:16.416563 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:16.416175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45921 (local address 127.0.0.1:46846)
0809 05:00:16.416333 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:16.416408 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 05:00:16.440399 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 24292
I20260809 05:00:16.455421 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:16.441455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.29.243.195:38481)
0809 05:00:16.441669 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:16.441676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:16.441694 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:00:16.441747 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:16.441750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:16.441827 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:16.441914 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:16.443384 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:16.444576 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:16.445910 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:16.446400 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:16.446521 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:16.449046 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:16.449060 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:16.449089 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:16.449135 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:16.452454 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:16.453149 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:16.453157 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:16.453160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:16.453348 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:16.453756 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:16.453760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:16.453762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:16.454189 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:16.454511 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:16.454795 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 05:00:16.455001 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:00:16.456221 24494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:00:16.497454 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:16.496845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45921 (local address 127.0.0.1:46850)
0809 05:00:16.497140 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:16.497258 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260809 05:00:16.498792 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:16.498302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:45921 (local address 127.0.0.1:46858)
0809 05:00:16.498528 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:16.498619 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260809 05:00:16.580775 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 24292
I20260809 05:00:16.605683 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 23708
I20260809 05:00:16.606932 23809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:00:16.719398 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 23708
I20260809 05:00:36.748522 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44193
--webserver_interface=127.29.243.254
--webserver_port=36065
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:44193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:00:37.103260 24582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:37.103628 24582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:37.103771 24582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:37.110936 24582 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 05:00:37.111043 24582 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 05:00:37.111099 24582 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 05:00:37.111168 24582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:00:37.111241 24582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:37.111354 24582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:37.111420 24582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:00:37.111500 24582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:00:37.121445 24582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:44193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:44193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=36065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:37.122754 24582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:37.124198 24582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:37.135753 24588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:37.135721 24589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:37.135685 24592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:37.136502 24582 server_base.cc:1034] running on GCE node
I20260809 05:00:37.137274 24582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:37.138626 24582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:37.139869 24582 hybrid_clock.cc:648] HybridClock initialized: now 1786251637139779 us; error 43 us; skew 500 ppm
I20260809 05:00:37.140252 24582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:37 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251637, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:37.148205 24582 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:00:37.149758 24582 webserver.cc:460] Webserver started at http://127.29.243.254:36065/ using document root <none> and password file <none>
I20260809 05:00:37.150455 24582 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:37.150555 24582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:37.156359 24582 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 05:00:37.159384 24601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:37.160889 24582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:00:37.161068 24582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "aad9dec6049048ffa753fc48d6502af8"
format_stamp: "Formatted at 2026-08-09 04:59:51 on dist-test-slave-21b9"
server_key: "0249b63ff1049ac62a59c5958552afd6"
server_key_iv: "0ff8aa20b2c3fa5b83707b7db021c943"
server_key_version: "encryptionkey@0"
I20260809 05:00:37.162050 24582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:37.210987 24582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:37.215651 24582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:37.215942 24582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:37.233809 24582 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:44193
I20260809 05:00:37.233858 24678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:44193 every 8 connection(s)
I20260809 05:00:37.235327 24582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:00:37.242409 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 24582
I20260809 05:00:37.242583 24679 sys_catalog.cc:263] Verifying existing consensus state
I20260809 05:00:37.243515 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:45921
--local_ip_for_outbound_sockets=127.29.243.193
--tserver_master_addrs=127.29.243.254:44193
--webserver_port=41307
--webserver_interface=127.29.243.193
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:00:37.247083 24679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Bootstrap starting.
I20260809 05:00:37.269773 24679 log.cc:824] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Log is configured to *not* fsync() on all Append() calls
I20260809 05:00:37.286852 24679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 05:00:37.287444 24679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Bootstrap complete.
I20260809 05:00:37.296448 24679 raft_consensus.cc:348] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 05:00:37.297310 24679 raft_consensus.cc:729] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad9dec6049048ffa753fc48d6502af8, State: Initialized, Role: FOLLOWER
I20260809 05:00:37.297955 24679 consensus_queue.cc:260] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 05:00:37.298182 24679 raft_consensus.cc:388] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:00:37.298296 24679 raft_consensus.cc:482] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:00:37.298465 24679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 2 FOLLOWER]: Advancing to term 3
I20260809 05:00:37.302184 24679 raft_consensus.cc:504] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 05:00:37.302698 24679 leader_election.cc:302] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aad9dec6049048ffa753fc48d6502af8; no voters:
I20260809 05:00:37.303385 24679 leader_election.cc:288] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 05:00:37.303591 24684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 05:00:37.304543 24684 raft_consensus.cc:686] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [term 3 LEADER]: Becoming Leader. State: Replica: aad9dec6049048ffa753fc48d6502af8, State: Running, Role: LEADER
I20260809 05:00:37.305173 24684 consensus_queue.cc:237] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } }
I20260809 05:00:37.305394 24679 sys_catalog.cc:564] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:00:37.307765 24687 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aad9dec6049048ffa753fc48d6502af8. Latest consensus state: current_term: 3 leader_uuid: "aad9dec6049048ffa753fc48d6502af8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } } }
I20260809 05:00:37.308154 24687 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: This master's current role is: LEADER
I20260809 05:00:37.308983 24686 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "aad9dec6049048ffa753fc48d6502af8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad9dec6049048ffa753fc48d6502af8" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 44193 } } }
I20260809 05:00:37.309441 24686 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8 [sys.catalog]: This master's current role is: LEADER
I20260809 05:00:37.320293 24698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:00:37.330976 24698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:00:37.332741 24698 catalog_manager.cc:1259] Loaded cluster ID: 12053e1ab25f498f9aa70e5ad201284c
I20260809 05:00:37.333205 24698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:00:37.337668 24698 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:00:37.339645 24698 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Loaded TSK: 0
I20260809 05:00:37.339886 24698 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aad9dec6049048ffa753fc48d6502af8: Loaded TSK: 1
F20260809 05:00:37.341143 24698 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0b86cd19ad google::LogMessage::Fail() at ??:0
@ 0x7f0b86cd488a google::LogMessage::SendToLog() at ??:0
@ 0x7f0b86cd14de google::LogMessage::Flush() at ??:0
@ 0x7f0b86cd2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0b8aa357d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0b8aa2473e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f0b9c7001c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0b9c6ff783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f0b9c78b2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f0b9c78b0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f0b9c7d55e9 std::function<>::operator()() at ??:0
@ 0x7f0b9c703cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0b9c701c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0b9c789610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f0b9c789439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0b8991d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0b8992091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f0b89920799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0b898fbeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0b8cbd56db start_thread at ??:0
@ 0x7f0b85b6b61f clone at ??:0
W20260809 05:00:37.651016 24682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:37.651345 24682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:37.651487 24682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:37.658809 24682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:37.658993 24682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:37.659155 24682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 05:00:37.669286 24682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:45921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=41307
--tserver_master_addrs=127.29.243.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:37.670658 24682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:37.672075 24682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:37.685171 24753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:37.685282 24754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:37.685318 24757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:37.686740 24682 server_base.cc:1034] running on GCE node
I20260809 05:00:37.687402 24682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:37.688313 24682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:37.689532 24682 hybrid_clock.cc:648] HybridClock initialized: now 1786251637689466 us; error 45 us; skew 500 ppm
I20260809 05:00:37.689848 24682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:37 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251637, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:37.693012 24682 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 05:00:37.694433 24682 webserver.cc:460] Webserver started at http://127.29.243.193:41307/ using document root <none> and password file <none>
I20260809 05:00:37.695132 24682 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:37.695253 24682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:37.700740 24682 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 05:00:37.703728 24768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:37.705071 24682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 05:00:37.705231 24682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "e30cd81995b940da8add19b81be9081d"
format_stamp: "Formatted at 2026-08-09 04:59:51 on dist-test-slave-21b9"
server_key: "6448269b4d177f1c7c0ffaebc5a5c0af"
server_key_iv: "21a983c86167af4df0c1e0a8a7c45609"
server_key_version: "encryptionkey@0"
I20260809 05:00:37.706245 24682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:37.728685 24682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:37.733018 24682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:37.733274 24682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:37.734279 24682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:00:37.736089 24777 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 05:00:37.741876 24682 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 05:00:37.742004 24682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260809 05:00:37.742129 24682 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 05:00:37.745326 24682 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 05:00:37.745414 24682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.001s
I20260809 05:00:37.745745 24777 tablet_bootstrap.cc:492] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Bootstrap starting.
I20260809 05:00:37.772475 24777 log.cc:824] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Log is configured to *not* fsync() on all Append() calls
I20260809 05:00:37.781569 24682 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:45921
I20260809 05:00:37.781594 24939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:45921 every 8 connection(s)
I20260809 05:00:37.783200 24682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 05:00:37.787642 24777 tablet_bootstrap.cc:492] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 05:00:37.787787 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 24682
I20260809 05:00:37.788185 24777 tablet_bootstrap.cc:492] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Bootstrap complete.
I20260809 05:00:37.788604 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:42213
--local_ip_for_outbound_sockets=127.29.243.194
--tserver_master_addrs=127.29.243.254:44193
--webserver_port=39497
--webserver_interface=127.29.243.194
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:00:37.788694 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:37.786204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:33785)
0809 05:00:37.787046 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:37.787392 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":393,"threads_started":1}
W20260809 05:00:37.789546 24940 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:44193 (0 consecutive failures): Network error: Failed to ping master at 127.29.243.254:44193: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
I20260809 05:00:37.789649 24777 ts_tablet_manager.cc:1397] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Time spent bootstrapping tablet: real 0.044s user 0.031s sys 0.010s
I20260809 05:00:37.790509 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:37.790117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:37985)
0809 05:00:37.790284 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:37.790372 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 05:00:37.791567 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:37.791118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:49833)
0809 05:00:37.791263 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:37.791388 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 05:00:37.791827 24940 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 05:00:37.796329 24777 raft_consensus.cc:348] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:37.797190 24777 raft_consensus.cc:729] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30cd81995b940da8add19b81be9081d, State: Initialized, Role: FOLLOWER
I20260809 05:00:37.797789 24777 consensus_queue.cc:260] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:37.799255 24777 ts_tablet_manager.cc:1428] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d: Time spent starting tablet: real 0.009s user 0.007s sys 0.004s
I20260809 05:00:37.799463 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:37.799006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:44579)
0809 05:00:37.799167 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:37.799260 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 05:00:37.800345 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:37.799991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:49007)
0809 05:00:37.800148 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:37.800216 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 05:00:38.132244 24944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:38.132571 24944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:38.132720 24944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:38.140275 24944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:38.140414 24944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:38.140594 24944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 05:00:38.150372 24944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:42213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=39497
--tserver_master_addrs=127.29.243.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:38.151700 24944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:38.153044 24944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:38.166286 24960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:38.166486 24962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:38.166610 24959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:38.167413 24944 server_base.cc:1034] running on GCE node
I20260809 05:00:38.168057 24944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:38.169077 24944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:38.170306 24944 hybrid_clock.cc:648] HybridClock initialized: now 1786251638170249 us; error 35 us; skew 500 ppm
I20260809 05:00:38.170657 24944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:38 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251638, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:38.173941 24944 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 05:00:38.175514 24944 webserver.cc:460] Webserver started at http://127.29.243.194:39497/ using document root <none> and password file <none>
I20260809 05:00:38.176211 24944 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:38.176337 24944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:38.182062 24944 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 05:00:38.184970 24972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:38.186318 24944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:00:38.186504 24944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "56e96fe9ef0a464e91582c6bfe6bb077"
format_stamp: "Formatted at 2026-08-09 04:59:52 on dist-test-slave-21b9"
server_key: "a87e684cd0e93444d8700bf34da36f80"
server_key_iv: "52a0e759d16fc31578b843027dae97b2"
server_key_version: "encryptionkey@0"
I20260809 05:00:38.187538 24944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:38.203720 24944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:38.207885 24944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:38.208137 24944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:38.209172 24944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:00:38.210705 24944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:00:38.210829 24944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:38.210939 24944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:00:38.211024 24944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:38.242479 24944 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:42213
I20260809 05:00:38.242547 25143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:42213 every 8 connection(s)
I20260809 05:00:38.243994 24944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 05:00:38.248807 25148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:38.246951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:59261)
0809 05:00:38.247721 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:38.247943 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":338,"threads_started":1}
I20260809 05:00:38.249425 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 24944
W20260809 05:00:38.249536 25144 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:44193 (0 consecutive failures): Network error: Failed to ping master at 127.29.243.254:44193: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
I20260809 05:00:38.250052 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:38481
--local_ip_for_outbound_sockets=127.29.243.195
--tserver_master_addrs=127.29.243.254:44193
--webserver_port=33689
--webserver_interface=127.29.243.195
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:00:38.250372 25148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:38.250018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:60239)
0809 05:00:38.250168 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:38.250255 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 05:00:38.251243 25148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:38.250914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:55573)
0809 05:00:38.251060 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:38.251137 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 05:00:38.251526 25144 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 05:00:38.596320 25151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:38.596602 25151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:38.596698 25151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:38.603865 25151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:38.603966 25151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:38.604084 25151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 05:00:38.613727 25151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:38481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=33689
--tserver_master_addrs=127.29.243.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:38.614987 25151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:38.616348 25151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:38.629365 25162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:38.629398 25166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:38.629338 25163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:38.630229 25151 server_base.cc:1034] running on GCE node
I20260809 05:00:38.630811 25151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:38.631783 25151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:38.632988 25151 hybrid_clock.cc:648] HybridClock initialized: now 1786251638632943 us; error 34 us; skew 500 ppm
I20260809 05:00:38.633291 25151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:38 dist-test-slave-21b9 krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251638, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:38.636348 25151 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 05:00:38.637823 25151 webserver.cc:460] Webserver started at http://127.29.243.195:33689/ using document root <none> and password file <none>
I20260809 05:00:38.638537 25151 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:38.638669 25151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:38.644102 25151 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:00:38.646836 25180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:38.648281 25151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:00:38.648447 25151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "a16fb27521a24539a0fa4215bb03cd8b"
format_stamp: "Formatted at 2026-08-09 04:59:52 on dist-test-slave-21b9"
server_key: "af8d1b521cd247c7c4d4c0666dd7d7b4"
server_key_iv: "275dda783bd733f3bbc041bbb9a74d7c"
server_key_version: "encryptionkey@0"
I20260809 05:00:38.649652 25151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:38.664758 25151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:38.668866 25151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:38.669145 25151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:38.670140 25151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:00:38.671543 25151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:00:38.671634 25151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:38.671736 25151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:00:38.671802 25151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:38.703575 25151 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:38481
I20260809 05:00:38.703662 25358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:38481 every 8 connection(s)
I20260809 05:00:38.705070 25151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 05:00:38.710052 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:38.708168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:44187)
0809 05:00:38.708882 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:38.709134 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":256,"threads_started":1}
I20260809 05:00:38.710177 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 25151
W20260809 05:00:38.710704 25360 heartbeater.cc:644] Failed to heartbeat to 127.29.243.254:44193 (0 consecutive failures): Network error: Failed to ping master at 127.29.243.254:44193: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
I20260809 05:00:38.711603 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:38.711209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:55663)
0809 05:00:38.711378 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:38.711467 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 05:00:38.712445 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:38.712150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:59253)
0809 05:00:38.712271 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:38.712341 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 05:00:38.712683 25360 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 05:00:38.801981 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:38.801203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:60091)
0809 05:00:38.801678 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:38.801777 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":228,"threads_started":1}
I20260809 05:00:39.124856 25366 raft_consensus.cc:482] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:00:39.125197 25366 raft_consensus.cc:504] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:39.126995 25366 leader_election.cc:288] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213), a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481)
Aug 09 05:00:39 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251637, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
Aug 09 05:00:39 dist-test-slave-21b9 krb5kdc[22796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251637, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 05:00:39.148360 25368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:39.127625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:49569 (local address 127.29.243.194:42213)
0809 05:00:39.129366 (+ 1741us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:39.129385 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:39.129428 (+ 43us) server_negotiation.cc:407] Connection header received
0809 05:00:39.131038 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:39.131048 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:39.131457 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:39.131730 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:39.133853 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:39.135020 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:39.136773 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:39.137379 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:39.137552 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:39.141680 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:39.141687 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:39.141698 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:39.141733 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:39.144728 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:39.145513 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:39.145521 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:39.145528 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:39.145720 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:39.146266 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:39.146271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:39.146274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:39.146853 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:39.147037 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:39.147613 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 05:00:39.147965 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1534,"thread_start_us":1400,"threads_started":1}
I20260809 05:00:39.148373 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:39.127842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:47093 (local address 127.29.243.195:38481)
0809 05:00:39.130063 (+ 2221us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:39.130074 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:39.130106 (+ 32us) server_negotiation.cc:407] Connection header received
0809 05:00:39.131576 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:39.131585 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:39.132030 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:39.132356 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:39.133844 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:39.135032 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:39.136773 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:39.137384 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:39.137551 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:39.141317 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:39.141329 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:39.141348 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:39.141400 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:39.144274 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:39.145145 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:39.145154 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:39.145161 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:39.145361 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:39.145928 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:39.145934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:39.145936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:39.146559 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:39.146711 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:39.147617 (+ 906us) server_negotiation.cc:299] Negotiation successful
0809 05:00:39.147959 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2063,"thread_start_us":1883,"threads_started":1}
I20260809 05:00:39.148379 25367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:39.127575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:38481 (local address 127.29.243.193:47093)
0809 05:00:39.129643 (+ 2068us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:39.129663 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:39.129805 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:39.132319 (+ 2514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:39.132322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:39.132884 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:00:39.133657 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:39.133678 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:39.135204 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:39.135208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:39.136636 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:39.136649 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:39.137586 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:39.137593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:39.137868 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:39.138600 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:00:39.138617 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:00:39.141016 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:00:39.144419 (+ 3403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:39.144429 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:39.144444 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:39.145001 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:39.145490 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:39.145495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:39.145498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:39.145754 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:39.146713 (+ 959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:00:39.146721 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:00:39.147094 (+ 373us) client_negotiation.cc:769] Sending connection context
0809 05:00:39.147364 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 05:00:39.147921 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1944,"mutex_wait_us":156,"spinlock_wait_cycles":1280,"thread_start_us":1775,"threads_started":1}
I20260809 05:00:39.148451 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:39.127527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:42213 (local address 127.29.243.193:49569)
0809 05:00:39.127877 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:39.127928 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:39.128553 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:39.131947 (+ 3394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:39.131959 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:39.132898 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:00:39.133657 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:39.133679 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:39.135209 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:39.135213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:39.136635 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:39.136648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:39.137637 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:39.137646 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:39.137848 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:39.138539 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:00:39.138562 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:00:39.141424 (+ 2862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:00:39.144865 (+ 3441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:39.144869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:39.144871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:39.145323 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:39.145913 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:39.145919 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:39.145922 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:39.146129 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:39.146977 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:00:39.146981 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:00:39.147102 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 05:00:39.147364 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 05:00:39.148055 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":393,"spinlock_wait_cycles":19456}
W20260809 05:00:39.151966 24772 leader_election.cc:341] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481): Not found: Tablet not found: cb6af646723541959493821475fa3600
W20260809 05:00:39.152257 24771 leader_election.cc:341] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213): Not found: Tablet not found: cb6af646723541959493821475fa3600
I20260809 05:00:39.152381 24771 leader_election.cc:302] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e30cd81995b940da8add19b81be9081d; no voters: 56e96fe9ef0a464e91582c6bfe6bb077, a16fb27521a24539a0fa4215bb03cd8b
I20260809 05:00:39.152791 25366 raft_consensus.cc:2726] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 05:00:39.253007 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:39.252086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:32827)
0809 05:00:39.252591 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:39.252743 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":229,"threads_started":1}
I20260809 05:00:39.715073 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:39.713376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:51745)
0809 05:00:39.714751 (+ 1375us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:39.714867 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1236,"thread_start_us":1119,"threads_started":1}
I20260809 05:00:39.803778 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:39.802965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:57937)
0809 05:00:39.803485 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:39.803604 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":256,"threads_started":1}
I20260809 05:00:40.254602 25377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:40.253882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:44293)
0809 05:00:40.254333 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:40.254414 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":251,"threads_started":1}
I20260809 05:00:40.716802 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:40.716017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:32819)
0809 05:00:40.716509 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:40.716611 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":237,"threads_started":1}
I20260809 05:00:40.805568 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:40.804706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:48973)
0809 05:00:40.805264 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:40.805352 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":247,"threads_started":1}
I20260809 05:00:41.256224 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:41.255503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:45955)
0809 05:00:41.255964 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:41.256052 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":236,"threads_started":1}
I20260809 05:00:41.281438 25382 raft_consensus.cc:482] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:00:41.281631 25382 raft_consensus.cc:504] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:41.282295 25382 leader_election.cc:288] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213), a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481)
W20260809 05:00:41.283344 24772 leader_election.cc:341] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481): Not found: Tablet not found: cb6af646723541959493821475fa3600
W20260809 05:00:41.283556 24771 leader_election.cc:341] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213): Not found: Tablet not found: cb6af646723541959493821475fa3600
I20260809 05:00:41.283682 24771 leader_election.cc:302] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e30cd81995b940da8add19b81be9081d; no voters: 56e96fe9ef0a464e91582c6bfe6bb077, a16fb27521a24539a0fa4215bb03cd8b
I20260809 05:00:41.283942 25382 raft_consensus.cc:2726] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 05:00:41.718366 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:41.717616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:58009)
0809 05:00:41.718109 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:41.718202 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":267,"threads_started":1}
I20260809 05:00:41.807188 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:41.806468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:53031)
0809 05:00:41.806936 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:41.807021 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":234,"threads_started":1}
I20260809 05:00:42.257943 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:42.257164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:41177)
0809 05:00:42.257671 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:42.257757 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":277,"threads_started":1}
I20260809 05:00:42.720144 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:42.719242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:36603)
0809 05:00:42.719809 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:42.719926 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":273,"threads_started":1}
I20260809 05:00:42.809161 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:42.808375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:41991)
0809 05:00:42.808851 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:42.808946 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":237,"threads_started":1}
I20260809 05:00:43.259585 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:43.258848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:51841)
0809 05:00:43.259296 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:43.259398 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":220,"threads_started":1}
I20260809 05:00:43.721983 25407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:43.721199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:54129)
0809 05:00:43.721694 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:43.721784 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":281,"threads_started":1}
I20260809 05:00:43.811420 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:43.810340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:38995)
0809 05:00:43.811038 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:43.811156 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":436,"threads_started":1}
I20260809 05:00:44.261224 25409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:44.260445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:53099)
0809 05:00:44.260944 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:44.261043 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":257,"threads_started":1}
I20260809 05:00:44.309681 25410 raft_consensus.cc:482] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:00:44.309921 25410 raft_consensus.cc:504] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:44.310690 25410 leader_election.cc:288] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213), a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481)
W20260809 05:00:44.311769 24772 leader_election.cc:341] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481): Not found: Tablet not found: cb6af646723541959493821475fa3600
W20260809 05:00:44.311972 24771 leader_election.cc:341] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213): Not found: Tablet not found: cb6af646723541959493821475fa3600
I20260809 05:00:44.312098 24771 leader_election.cc:302] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e30cd81995b940da8add19b81be9081d; no voters: 56e96fe9ef0a464e91582c6bfe6bb077, a16fb27521a24539a0fa4215bb03cd8b
I20260809 05:00:44.312353 25410 raft_consensus.cc:2726] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 05:00:44.723729 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:44.722887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:44891)
0809 05:00:44.723422 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:44.723508 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":275,"threads_started":1}
I20260809 05:00:44.813167 25412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:44.812318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:48377)
0809 05:00:44.812843 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:44.812971 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":282,"threads_started":1}
I20260809 05:00:45.262998 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:45.262206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:35483)
0809 05:00:45.262723 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:45.262810 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":222,"threads_started":1}
I20260809 05:00:45.725531 25414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:45.724645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:36827)
0809 05:00:45.725189 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:45.725273 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":234,"threads_started":1}
I20260809 05:00:45.814793 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:45.814065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:36351)
0809 05:00:45.814536 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:45.814622 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":240,"threads_started":1}
I20260809 05:00:46.041319 25418 raft_consensus.cc:482] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:00:46.041523 25418 raft_consensus.cc:504] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:46.042169 25418 leader_election.cc:288] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213), a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481)
W20260809 05:00:46.043171 24772 leader_election.cc:341] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481): Not found: Tablet not found: cb6af646723541959493821475fa3600
W20260809 05:00:46.043413 24771 leader_election.cc:341] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213): Not found: Tablet not found: cb6af646723541959493821475fa3600
I20260809 05:00:46.043519 24771 leader_election.cc:302] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e30cd81995b940da8add19b81be9081d; no voters: 56e96fe9ef0a464e91582c6bfe6bb077, a16fb27521a24539a0fa4215bb03cd8b
I20260809 05:00:46.043780 25418 raft_consensus.cc:2726] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 05:00:46.264746 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:46.263951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:56163)
0809 05:00:46.264449 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:46.264548 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":249,"threads_started":1}
I20260809 05:00:46.727380 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:46.726508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:34449)
0809 05:00:46.727008 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:46.727106 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":249,"threads_started":1}
I20260809 05:00:46.816442 25422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:46.815703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:43911)
0809 05:00:46.816155 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:46.816251 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":242,"threads_started":1}
I20260809 05:00:47.266471 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:47.265724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:52403)
0809 05:00:47.266215 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:47.266299 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":232,"threads_started":1}
I20260809 05:00:47.729128 25425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:47.728356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:35013)
0809 05:00:47.728852 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:47.728947 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":235,"threads_started":1}
I20260809 05:00:47.818437 25427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:47.817495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:34353)
0809 05:00:47.818078 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:47.818198 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":326,"threads_started":1}
I20260809 05:00:48.268227 25428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:48.267474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:41343)
0809 05:00:48.267947 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:48.268042 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":260,"threads_started":1}
I20260809 05:00:48.730880 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:48.730113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:59501)
0809 05:00:48.730565 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:48.730649 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":246,"threads_started":1}
I20260809 05:00:48.820235 25430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:48.819420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:43869)
0809 05:00:48.819942 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:48.820043 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":288,"threads_started":1}
I20260809 05:00:49.269892 25431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:49.269109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:54769)
0809 05:00:49.269632 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:49.269727 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":248,"threads_started":1}
I20260809 05:00:49.732676 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:49.731887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:47331)
0809 05:00:49.732413 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:49.732506 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":256,"threads_started":1}
I20260809 05:00:49.821926 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:49.821126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:55453)
0809 05:00:49.821649 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:49.821741 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":302,"threads_started":1}
I20260809 05:00:50.262045 25435 raft_consensus.cc:482] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:00:50.262253 25435 raft_consensus.cc:504] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e30cd81995b940da8add19b81be9081d" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 45921 } } peers { permanent_uuid: "56e96fe9ef0a464e91582c6bfe6bb077" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42213 } } peers { permanent_uuid: "a16fb27521a24539a0fa4215bb03cd8b" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38481 } }
I20260809 05:00:50.263007 25435 leader_election.cc:288] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213), a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481)
W20260809 05:00:50.264030 24772 leader_election.cc:341] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a16fb27521a24539a0fa4215bb03cd8b (127.29.243.195:38481): Not found: Tablet not found: cb6af646723541959493821475fa3600
W20260809 05:00:50.264201 24771 leader_election.cc:341] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 56e96fe9ef0a464e91582c6bfe6bb077 (127.29.243.194:42213): Not found: Tablet not found: cb6af646723541959493821475fa3600
I20260809 05:00:50.264302 24771 leader_election.cc:302] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e30cd81995b940da8add19b81be9081d; no voters: 56e96fe9ef0a464e91582c6bfe6bb077, a16fb27521a24539a0fa4215bb03cd8b
I20260809 05:00:50.264531 25435 raft_consensus.cc:2726] T cb6af646723541959493821475fa3600 P e30cd81995b940da8add19b81be9081d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 05:00:50.271425 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:50.270715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:49153)
0809 05:00:50.271141 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:50.271221 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":213,"threads_started":1}
I20260809 05:00:50.734494 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:50.733673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:34449)
0809 05:00:50.734152 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:50.734258 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":247,"threads_started":1}
I20260809 05:00:50.823701 25439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:50.822869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:51987)
0809 05:00:50.823401 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:50.823511 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":249,"threads_started":1}
I20260809 05:00:51.273152 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:51.272397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:51263)
0809 05:00:51.272886 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:51.272983 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":241,"threads_started":1}
I20260809 05:00:51.736255 25441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:51.735522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:54561)
0809 05:00:51.735988 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:51.736081 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":228,"threads_started":1}
I20260809 05:00:51.825426 25442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:51.824610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:44121)
0809 05:00:51.825120 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:51.825210 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":255,"threads_started":1}
I20260809 05:00:52.274921 25443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:52.274096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:50167)
0809 05:00:52.274598 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:52.274698 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":252,"threads_started":1}
I20260809 05:00:52.737984 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:52.737183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:34527)
0809 05:00:52.737679 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:52.737778 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":241,"threads_started":1}
I20260809 05:00:52.827428 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:52.826588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.193:40015)
0809 05:00:52.827078 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:52.827185 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":243,"threads_started":1}
I20260809 05:00:53.276888 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:53.276129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.194:55331)
0809 05:00:53.276585 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:53.276716 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":219,"threads_started":1}
I20260809 05:00:53.714670 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 24682
I20260809 05:00:53.729334 25448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:53.715898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.29.243.193:45921)
0809 05:00:53.716411 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:53.716418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:53.716447 (+ 29us) server_negotiation.cc:407] Connection header received
0809 05:00:53.716516 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:53.716523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:53.716749 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:53.716905 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:53.718365 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:53.719419 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:53.720660 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:53.721202 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:53.721298 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:53.724048 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:53.724060 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:53.724076 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:53.724109 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:53.726890 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:53.727536 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:53.727543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:53.727546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:53.727684 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:53.727979 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:53.727982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:53.727984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:53.728317 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:53.728568 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:53.728836 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 05:00:53.729052 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":224,"threads_started":1}
I20260809 05:00:53.730664 24935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:00:53.739980 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:53.739177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:44193 (local address 127.29.243.195:39009)
0809 05:00:53.739698 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:53.739826 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.254:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":266,"threads_started":1}
I20260809 05:00:53.820052 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 24682
I20260809 05:00:53.852952 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 24944
I20260809 05:00:53.866266 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:53.853765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.29.243.194:42213)
0809 05:00:53.854262 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:53.854267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:53.854282 (+ 15us) server_negotiation.cc:407] Connection header received
0809 05:00:53.854339 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:53.854342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:53.854412 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:53.854538 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:53.855970 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:53.856739 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:53.857982 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:53.858429 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:53.858537 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:53.861143 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:53.861148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:53.861151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:53.861185 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:53.863636 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:53.864313 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:53.864317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:53.864319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:53.864465 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:53.864829 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:53.864833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:53.864835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:53.865107 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:53.865365 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:53.865674 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 05:00:53.865900 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":248,"threads_started":1}
I20260809 05:00:53.867077 25138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:00:53.950742 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 24944
I20260809 05:00:53.977645 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 25151
I20260809 05:00:53.991250 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:53.978436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.29.243.195:38481)
0809 05:00:53.979008 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:53.979013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:53.979030 (+ 17us) server_negotiation.cc:407] Connection header received
0809 05:00:53.979088 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:53.979091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:53.979163 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:53.979295 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:53.980731 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:53.981536 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:53.982811 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:53.983244 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:53.983487 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:53.986158 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:53.986164 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:53.986167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:53.986202 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:53.988816 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:53.989567 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:53.989572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:53.989574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:53.989729 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:53.990029 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:53.990032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:53.990033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:53.990284 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:53.990500 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:53.990757 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 05:00:53.990928 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":281,"threads_started":1}
I20260809 05:00:53.992033 25354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:00:54.077936 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 25151
2026-08-09T05:00:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64964 ms)
[----------] 30 tests from SecurityITest (306195 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:00:54 dist-test-slave-21b9 krb5kdc[25461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:00:54 dist-test-slave-21b9 krb5kdc[25461](info): set up 2 sockets
Aug 09 05:00:54 dist-test-slave-21b9 krb5kdc[25461](info): commencing operation
krb5kdc: starting...
W20260809 05:00:55.332216 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:00:55 dist-test-slave-21b9 krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:00:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:00:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 05:00:55.636548 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:43681
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:43681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:00:55.978940 25484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:55.979228 25484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:55.979373 25484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:55.986469 25484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:00:55.986569 25484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:55.986626 25484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:55.986708 25484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:00:55.986763 25484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:00:55.996380 25484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:43681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:43681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:55.997721 25484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:55.999049 25484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:56.011214 25496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:56.011301 25492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:56.011489 25491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:56.013244 25484 server_base.cc:1034] running on GCE node
I20260809 05:00:56.014302 25484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:56.016338 25484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:56.017570 25484 hybrid_clock.cc:648] HybridClock initialized: now 1786251656017505 us; error 54 us; skew 500 ppm
I20260809 05:00:56.017972 25484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:56 dist-test-slave-21b9 krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251656, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:56.024315 25484 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:00:56.025961 25484 webserver.cc:460] Webserver started at http://127.29.243.254:41261/ using document root <none> and password file <none>
I20260809 05:00:56.026686 25484 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:56.026809 25484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:56.027125 25484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:00:56.030249 25484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "e0e095632f48474191d7a7873ea1801d"
format_stamp: "Formatted at 2026-08-09 05:00:56 on dist-test-slave-21b9"
server_key: "381e7d00f2c8aa684cb0883be97ce662"
server_key_iv: "f661262073d7cd16cfee71c1d9d2815e"
server_key_version: "encryptionkey@0"
I20260809 05:00:56.030997 25484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "e0e095632f48474191d7a7873ea1801d"
format_stamp: "Formatted at 2026-08-09 05:00:56 on dist-test-slave-21b9"
server_key: "381e7d00f2c8aa684cb0883be97ce662"
server_key_iv: "f661262073d7cd16cfee71c1d9d2815e"
server_key_version: "encryptionkey@0"
I20260809 05:00:56.036433 25484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 05:00:56.039961 25509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:56.041291 25484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:00:56.041455 25484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "e0e095632f48474191d7a7873ea1801d"
format_stamp: "Formatted at 2026-08-09 05:00:56 on dist-test-slave-21b9"
server_key: "381e7d00f2c8aa684cb0883be97ce662"
server_key_iv: "f661262073d7cd16cfee71c1d9d2815e"
server_key_version: "encryptionkey@0"
I20260809 05:00:56.041652 25484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:56.065701 25484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:56.070071 25484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:56.070344 25484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:56.087845 25484 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:43681
I20260809 05:00:56.087868 25596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:43681 every 8 connection(s)
I20260809 05:00:56.089284 25484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:00:56.093152 25597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:00:56.096721 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 25484
I20260809 05:00:56.096942 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:00:56.097816 30671 external_mini_cluster.cc:1428] Setting key 1234572ad8e28042669aa211c356cc48
I20260809 05:00:56.106011 25597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d: Bootstrap starting.
I20260809 05:00:56.109128 25597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d: Neither blocks nor log segments found. Creating new log.
I20260809 05:00:56.110111 25597 log.cc:824] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d: Log is configured to *not* fsync() on all Append() calls
Aug 09 05:00:56 dist-test-slave-21b9 krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:00:56.113103 25597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d: No bootstrap required, opened a new log
I20260809 05:00:56.120579 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:56.100131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56402 (local address 127.29.243.254:43681)
0809 05:00:56.100946 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:56.100959 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:56.101005 (+ 46us) server_negotiation.cc:407] Connection header received
0809 05:00:56.102660 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:56.102685 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:56.103118 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:56.103555 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:56.105073 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:56.106448 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:56.107789 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:56.108476 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:56.108643 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:56.113143 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:56.113153 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:56.113174 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:56.113219 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:56.116325 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:56.117059 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:56.117066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:56.117074 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:56.117263 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:56.117672 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:56.117676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:56.117678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:56.118251 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:56.118543 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:56.119072 (+ 529us) server_negotiation.cc:299] Negotiation successful
0809 05:00:56.119513 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":371,"threads_started":1}
I20260809 05:00:56.120851 25597 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e095632f48474191d7a7873ea1801d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43681 } }
I20260809 05:00:56.121219 25597 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:00:56.121315 25597 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e095632f48474191d7a7873ea1801d, State: Initialized, Role: FOLLOWER
I20260809 05:00:56.121857 25597 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e095632f48474191d7a7873ea1801d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43681 } }
I20260809 05:00:56.122108 25597 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:00:56.122237 25597 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:00:56.122396 25597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:00:56.124847 25597 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e095632f48474191d7a7873ea1801d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43681 } }
I20260809 05:00:56.125312 25597 leader_election.cc:302] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0e095632f48474191d7a7873ea1801d; no voters:
I20260809 05:00:56.125901 25597 leader_election.cc:288] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:00:56.126000 25609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:00:56.127044 25609 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [term 1 LEADER]: Becoming Leader. State: Replica: e0e095632f48474191d7a7873ea1801d, State: Running, Role: LEADER
I20260809 05:00:56.127641 25609 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e095632f48474191d7a7873ea1801d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43681 } }
I20260809 05:00:56.128008 25597 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:00:56.130477 25612 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0e095632f48474191d7a7873ea1801d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e095632f48474191d7a7873ea1801d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43681 } } }
I20260809 05:00:56.130527 25613 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0e095632f48474191d7a7873ea1801d. Latest consensus state: current_term: 1 leader_uuid: "e0e095632f48474191d7a7873ea1801d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e095632f48474191d7a7873ea1801d" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43681 } } }
I20260809 05:00:56.130903 25613 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [sys.catalog]: This master's current role is: LEADER
I20260809 05:00:56.130904 25612 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d [sys.catalog]: This master's current role is: LEADER
I20260809 05:00:56.133526 25621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:00:56.138001 25621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:00:56.146826 25621 catalog_manager.cc:1347] Generated new cluster ID: 585aec4f30ed49c783d02989fcbf255c
I20260809 05:00:56.146943 25621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:00:56.164732 25621 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:00:56.166601 25621 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:00:56.175042 25621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d: Generated new TSK 0
I20260809 05:00:56.175894 25621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 05:00:56.257233 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43681
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:00:56.610708 25643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:56.610982 25643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:56.611122 25643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:56.618443 25643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:56.618546 25643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:56.618695 25643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 05:00:56.628655 25643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:56.630097 25643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:56.631573 25643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:56.644297 25652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:56.644361 25650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:56.644330 25648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:56.645665 25643 server_base.cc:1034] running on GCE node
I20260809 05:00:56.646252 25643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:56.647213 25643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:56.648458 25643 hybrid_clock.cc:648] HybridClock initialized: now 1786251656648408 us; error 40 us; skew 500 ppm
I20260809 05:00:56.648768 25643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:56 dist-test-slave-21b9 krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251656, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:56.651907 25643 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 05:00:56.653288 25643 webserver.cc:460] Webserver started at http://127.29.243.193:44961/ using document root <none> and password file <none>
I20260809 05:00:56.653990 25643 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:56.654093 25643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:56.654361 25643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:00:56.657410 25643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "f71c28e285304820a5a85e1b5259161e"
format_stamp: "Formatted at 2026-08-09 05:00:56 on dist-test-slave-21b9"
server_key: "bbe6b0395dc7ff5ff09c480fcffffc71"
server_key_iv: "b4cd8bab997fdf84159569a242d04056"
server_key_version: "encryptionkey@0"
I20260809 05:00:56.658169 25643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "f71c28e285304820a5a85e1b5259161e"
format_stamp: "Formatted at 2026-08-09 05:00:56 on dist-test-slave-21b9"
server_key: "bbe6b0395dc7ff5ff09c480fcffffc71"
server_key_iv: "b4cd8bab997fdf84159569a242d04056"
server_key_version: "encryptionkey@0"
I20260809 05:00:56.663998 25643 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 05:00:56.667709 25663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:56.669138 25643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:00:56.669313 25643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "f71c28e285304820a5a85e1b5259161e"
format_stamp: "Formatted at 2026-08-09 05:00:56 on dist-test-slave-21b9"
server_key: "bbe6b0395dc7ff5ff09c480fcffffc71"
server_key_iv: "b4cd8bab997fdf84159569a242d04056"
server_key_version: "encryptionkey@0"
I20260809 05:00:56.669512 25643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:56.687433 25643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:56.692941 25643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:56.693244 25643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:56.694279 25643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:00:56.696106 25643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:00:56.696209 25643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:56.696367 25643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:00:56.696434 25643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:56.728718 25643 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:34587
I20260809 05:00:56.728804 25860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:34587 every 8 connection(s)
I20260809 05:00:56.730073 25643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 05:00:56.739181 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 25643
I20260809 05:00:56.739437 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 05:00:56.740291 30671 external_mini_cluster.cc:1428] Setting key 91cc9a1377edd575dab66225e5d5d65b
Aug 09 05:00:56 dist-test-slave-21b9 krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251656, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:00:56.750826 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:56.732688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:49935 (local address 127.29.243.254:43681)
0809 05:00:56.733120 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:56.733125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:56.734217 (+ 1092us) server_negotiation.cc:407] Connection header received
0809 05:00:56.736102 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:56.736107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:56.736187 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:56.736283 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:56.737757 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:56.738500 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:56.740801 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:56.741357 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:56.741489 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:56.745037 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:56.745042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:56.745046 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:56.745085 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:56.747907 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:56.748600 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:56.748604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:56.748607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:56.748750 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:56.749144 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:56.749149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:56.749151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:56.749472 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:56.749607 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:56.750271 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 05:00:56.750492 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":218,"threads_started":1}
I20260809 05:00:56.751700 25864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:56.733313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:43681 (local address 127.29.243.193:49935)
0809 05:00:56.734073 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:56.734113 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:56.735832 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:56.736521 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:56.736529 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:56.737038 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:00:56.737598 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:56.737615 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:56.738649 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:56.738653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:56.740656 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:56.740672 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:56.741597 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:56.741606 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:56.741853 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:56.742493 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:00:56.742516 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:00:56.744817 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:00:56.748050 (+ 3233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:56.748057 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:56.748070 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:56.748467 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:56.748834 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:56.748837 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:56.748840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:56.749026 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:56.749592 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:00:56.749598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:00:56.749902 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 05:00:56.750121 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 05:00:56.750511 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":283,"threads_started":1}
I20260809 05:00:56.753202 25861 heartbeater.cc:347] Connected to a master server at 127.29.243.254:43681
I20260809 05:00:56.753484 25861 heartbeater.cc:464] Registering TS with master...
I20260809 05:00:56.754195 25861 heartbeater.cc:511] Master 127.29.243.254:43681 requested a full tablet report, sending...
I20260809 05:00:56.756435 25536 ts_manager.cc:194] Registered new tserver with Master: f71c28e285304820a5a85e1b5259161e (127.29.243.193:34587)
I20260809 05:00:56.757913 25536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:49935
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 05:00:56.806188 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43681
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:00:57.150878 25869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:57.151168 25869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:57.151338 25869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:57.158317 25869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:57.158443 25869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:57.158613 25869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 05:00:57.168351 25869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:57.169780 25869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:57.171118 25869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:57.184182 25875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:57.184291 25877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:57.184717 25879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:57.185178 25869 server_base.cc:1034] running on GCE node
I20260809 05:00:57.185806 25869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:57.186712 25869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:57.187913 25869 hybrid_clock.cc:648] HybridClock initialized: now 1786251657187860 us; error 45 us; skew 500 ppm
I20260809 05:00:57.188230 25869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:57 dist-test-slave-21b9 krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251657, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:57.191454 25869 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 05:00:57.192945 25869 webserver.cc:460] Webserver started at http://127.29.243.194:40973/ using document root <none> and password file <none>
I20260809 05:00:57.193640 25869 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:57.193763 25869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:57.194083 25869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:00:57.197062 25869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "3694b7d1447041f88e62d03601a51dbc"
format_stamp: "Formatted at 2026-08-09 05:00:57 on dist-test-slave-21b9"
server_key: "5ae0b0ae548a1f51e28dffcbab45dba5"
server_key_iv: "810e5c8bb2b7c07836b9d316ddc4cf8e"
server_key_version: "encryptionkey@0"
I20260809 05:00:57.197808 25869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "3694b7d1447041f88e62d03601a51dbc"
format_stamp: "Formatted at 2026-08-09 05:00:57 on dist-test-slave-21b9"
server_key: "5ae0b0ae548a1f51e28dffcbab45dba5"
server_key_iv: "810e5c8bb2b7c07836b9d316ddc4cf8e"
server_key_version: "encryptionkey@0"
I20260809 05:00:57.203018 25869 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:00:57.206773 25891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:57.208171 25869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:00:57.208333 25869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "3694b7d1447041f88e62d03601a51dbc"
format_stamp: "Formatted at 2026-08-09 05:00:57 on dist-test-slave-21b9"
server_key: "5ae0b0ae548a1f51e28dffcbab45dba5"
server_key_iv: "810e5c8bb2b7c07836b9d316ddc4cf8e"
server_key_version: "encryptionkey@0"
I20260809 05:00:57.208524 25869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:57.226068 25869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:57.231487 25869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:57.231801 25869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:57.232905 25869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:00:57.234316 25869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:00:57.234403 25869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:57.234529 25869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:00:57.234586 25869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:57.265913 25869 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:34967
I20260809 05:00:57.265971 26088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:34967 every 8 connection(s)
I20260809 05:00:57.267189 25869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 05:00:57.275693 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 25869
I20260809 05:00:57.275900 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 05:00:57.276335 30671 external_mini_cluster.cc:1428] Setting key 70ca9a847ea0357bc8a7d5e1816ff18f
Aug 09 05:00:57 dist-test-slave-21b9 krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251657, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:00:57.287782 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:57.269745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:42397 (local address 127.29.243.254:43681)
0809 05:00:57.270156 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:57.270161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:57.271181 (+ 1020us) server_negotiation.cc:407] Connection header received
0809 05:00:57.273059 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:57.273065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:57.273159 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:57.273253 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:57.274818 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.275709 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.277434 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.278051 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.278171 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:57.281735 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:57.281742 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:57.281746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:57.281783 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:57.284756 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:57.285448 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:57.285453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:57.285456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:57.285605 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:57.286044 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:57.286048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:57.286050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:57.286400 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:57.286538 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:57.287200 (+ 662us) server_negotiation.cc:299] Negotiation successful
0809 05:00:57.287441 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":195,"threads_started":1}
I20260809 05:00:57.288781 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:57.270313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:43681 (local address 127.29.243.194:42397)
0809 05:00:57.271021 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:57.271059 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:57.272768 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:57.273526 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:57.273534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:57.274088 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:00:57.274637 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:57.274654 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.275893 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.275898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:57.277286 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:57.277302 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.278273 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.278279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:57.278518 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:57.279144 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:00:57.279167 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:00:57.281502 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:00:57.284907 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:57.284914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:57.284927 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:57.285335 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:57.285721 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:57.285725 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:57.285727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:57.285940 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:57.286512 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:00:57.286518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:00:57.286834 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 05:00:57.287079 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 05:00:57.287524 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":256,"threads_started":1}
I20260809 05:00:57.290230 26090 heartbeater.cc:347] Connected to a master server at 127.29.243.254:43681
I20260809 05:00:57.290526 26090 heartbeater.cc:464] Registering TS with master...
I20260809 05:00:57.291249 26090 heartbeater.cc:511] Master 127.29.243.254:43681 requested a full tablet report, sending...
I20260809 05:00:57.292703 25536 ts_manager.cc:194] Registered new tserver with Master: 3694b7d1447041f88e62d03601a51dbc (127.29.243.194:34967)
I20260809 05:00:57.293514 25536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:42397
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 05:00:57.342821 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43681
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:00:57.686905 26099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:57.687213 26099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:57.687390 26099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:57.694736 26099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:57.694868 26099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:57.695042 26099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 05:00:57.705065 26099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:43681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:57.706542 26099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:57.707990 26099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:57.721587 26105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:57.721657 26108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:57.721813 26112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:57.723012 26099 server_base.cc:1034] running on GCE node
I20260809 05:00:57.723747 26099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:57.724761 26099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:57.725996 26099 hybrid_clock.cc:648] HybridClock initialized: now 1786251657725929 us; error 47 us; skew 500 ppm
I20260809 05:00:57.726341 26099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:00:57 dist-test-slave-21b9 krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251657, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:00:57.730273 26099 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 05:00:57.732173 26099 webserver.cc:460] Webserver started at http://127.29.243.195:38729/ using document root <none> and password file <none>
I20260809 05:00:57.733116 26099 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:57.733264 26099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:57.733673 26099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:00:57.737011 26099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb"
format_stamp: "Formatted at 2026-08-09 05:00:57 on dist-test-slave-21b9"
server_key: "90aa55383c4f16aa77f93e048696ae62"
server_key_iv: "060b39e65c91c02d5c4293caa791de38"
server_key_version: "encryptionkey@0"
I20260809 05:00:57.737756 26099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb"
format_stamp: "Formatted at 2026-08-09 05:00:57 on dist-test-slave-21b9"
server_key: "90aa55383c4f16aa77f93e048696ae62"
server_key_iv: "060b39e65c91c02d5c4293caa791de38"
server_key_version: "encryptionkey@0"
I20260809 05:00:57.743201 26099 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:00:57.746990 26120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:57.748335 26099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:00:57.748487 26099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb"
format_stamp: "Formatted at 2026-08-09 05:00:57 on dist-test-slave-21b9"
server_key: "90aa55383c4f16aa77f93e048696ae62"
server_key_iv: "060b39e65c91c02d5c4293caa791de38"
server_key_version: "encryptionkey@0"
I20260809 05:00:57.748653 26099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:57.760720 25861 heartbeater.cc:503] Master 127.29.243.254:43681 was elected leader, sending a full tablet report...
I20260809 05:00:57.784228 26099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:57.788551 26099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:57.788833 26099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:57.789888 26099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:00:57.791237 26099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:00:57.791359 26099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 05:00:57.791486 26099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:00:57.791550 26099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:57.822486 26099 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:38527
I20260809 05:00:57.822549 26302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:38527 every 8 connection(s)
I20260809 05:00:57.823827 26099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 05:00:57.831740 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 26099
I20260809 05:00:57.831912 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 05:00:57.832360 30671 external_mini_cluster.cc:1428] Setting key ba807f1216653c805dd3142eacbc8448
Aug 09 05:00:57 dist-test-slave-21b9 krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251657, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:00:57.843732 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:57.826401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:40947 (local address 127.29.243.254:43681)
0809 05:00:57.826864 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:57.826871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:57.827937 (+ 1066us) server_negotiation.cc:407] Connection header received
0809 05:00:57.829864 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:57.829870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:57.829967 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:57.830064 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:57.831679 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.832683 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.834523 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.834933 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.835046 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:57.838463 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:57.838468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:57.838471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:57.838503 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:57.841079 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:57.841730 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:57.841733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:57.841735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:57.841850 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:57.842194 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:57.842196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:57.842198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:57.842485 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:57.842606 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:57.843219 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 05:00:57.843458 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":206,"threads_started":1}
I20260809 05:00:57.844624 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:57.827017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:43681 (local address 127.29.243.195:40947)
0809 05:00:57.827742 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:57.827803 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:57.829588 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:57.830313 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:57.830322 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:57.830820 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:00:57.831483 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:57.831520 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.832858 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.832862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:57.834395 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:57.834412 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.835159 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.835167 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:57.835477 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:57.836093 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:00:57.836116 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:00:57.838249 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:00:57.841202 (+ 2953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:57.841211 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:57.841227 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:57.841636 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:57.841925 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:00:57.841928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:00:57.841930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:00:57.842113 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:00:57.842585 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:00:57.842590 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:00:57.842883 (+ 293us) client_negotiation.cc:769] Sending connection context
0809 05:00:57.843116 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 05:00:57.843535 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":295,"threads_started":1}
I20260809 05:00:57.846263 26303 heartbeater.cc:347] Connected to a master server at 127.29.243.254:43681
I20260809 05:00:57.846592 26303 heartbeater.cc:464] Registering TS with master...
I20260809 05:00:57.847342 26303 heartbeater.cc:511] Master 127.29.243.254:43681 requested a full tablet report, sending...
I20260809 05:00:57.848738 25536 ts_manager.cc:194] Registered new tserver with Master: 28a195f8fde441ab9a5ec2aa0f8b16fb (127.29.243.195:38527)
I20260809 05:00:57.849526 25536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:40947
I20260809 05:00:57.859580 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:00:57.876248 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:57.864008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.29.243.254:43681)
0809 05:00:57.864159 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:57.864163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:57.864540 (+ 377us) server_negotiation.cc:407] Connection header received
0809 05:00:57.864691 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:57.864694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:57.864767 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:57.864842 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:57.866170 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.866865 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.868312 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.868699 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.868787 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:57.871122 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:57.871127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:57.871130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:57.871160 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:57.873725 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:57.874449 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:57.874453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:57.874455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:57.874587 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:57.874905 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:57.874908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:57.874910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:57.875207 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:57.875437 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:57.875855 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 05:00:57.876024 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 05:00:57.880352 25536 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:00:57.887189 25536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:00:57.908967 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:57.898846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:34587 (local address 127.0.0.1:45198)
0809 05:00:57.899826 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:57.899878 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:57.900053 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:57.900870 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:57.900877 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:57.900923 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:00:57.901252 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:57.901265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.903524 (+ 2259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.903529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:57.906026 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:57.906041 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.907983 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.907990 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:57.908198 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:57.908261 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 05:00:57.908371 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 05:00:57.908572 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":594,"threads_started":1}
I20260809 05:00:57.909713 26330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:57.899173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45198 (local address 127.29.243.193:34587)
0809 05:00:57.899834 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:57.899843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:57.899971 (+ 128us) server_negotiation.cc:407] Connection header received
0809 05:00:57.900329 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:57.900340 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:57.900618 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:57.901496 (+ 878us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:00:57.901738 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.903362 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.906221 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.907740 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.908920 (+ 1180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:57.909152 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:57.909233 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 05:00:57.909390 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":291,"threads_started":1}
I20260809 05:00:57.910192 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:57.900131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:38527 (local address 127.0.0.1:51596)
0809 05:00:57.900647 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:57.900668 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:57.900780 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:57.903223 (+ 2443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:57.903227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:57.903257 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:00:57.903788 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:57.903804 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.905606 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.905610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:57.907570 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:57.907584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.909374 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.909380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:57.909560 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:57.909594 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 05:00:57.909656 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 05:00:57.909786 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":1,"thread_start_us":282,"threads_started":1}
I20260809 05:00:57.910660 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:57.900720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:34967 (local address 127.0.0.1:33888)
0809 05:00:57.903266 (+ 2546us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:57.903303 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:57.903426 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:57.903990 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:57.903994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:57.904018 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:00:57.904424 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:57.904436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.906306 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.906311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:57.907814 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:57.907828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.909818 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.909825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:57.910001 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:57.910034 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 05:00:57.910133 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 05:00:57.910265 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2425,"mutex_wait_us":23,"thread_start_us":2269,"threads_started":1}
I20260809 05:00:57.910874 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:57.900518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51596 (local address 127.29.243.195:38527)
0809 05:00:57.902585 (+ 2067us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:57.902594 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:57.902629 (+ 35us) server_negotiation.cc:407] Connection header received
0809 05:00:57.902722 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:57.902730 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:57.902965 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:57.903228 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:00:57.903990 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.905462 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.907738 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.909148 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.910121 (+ 973us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:57.910301 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:57.910371 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 05:00:57.910530 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1899,"thread_start_us":1738,"threads_started":1}
I20260809 05:00:57.911365 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:57.901231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33888 (local address 127.29.243.194:34967)
0809 05:00:57.902836 (+ 1605us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:57.902843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:57.903546 (+ 703us) server_negotiation.cc:407] Connection header received
0809 05:00:57.903646 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:57.903651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:57.903803 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:57.903965 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:00:57.904738 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.906185 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.908023 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:57.909425 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:57.909575 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:57.909734 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:57.910847 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0809 05:00:57.911019 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1456,"thread_start_us":1350,"threads_started":1}
I20260809 05:00:57.913769 25749 tablet_service.cc:1467] Processing CreateTablet for tablet cd5d1737fe344f49857b25858415d6dc (DEFAULT_TABLE table=test-table [id=76441bfd9a0b4f29a38db534c55e9a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:00:57.914556 26195 tablet_service.cc:1467] Processing CreateTablet for tablet cd5d1737fe344f49857b25858415d6dc (DEFAULT_TABLE table=test-table [id=76441bfd9a0b4f29a38db534c55e9a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:00:57.915395 25749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd5d1737fe344f49857b25858415d6dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:00:57.915254 25967 tablet_service.cc:1467] Processing CreateTablet for tablet cd5d1737fe344f49857b25858415d6dc (DEFAULT_TABLE table=test-table [id=76441bfd9a0b4f29a38db534c55e9a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:00:57.916404 26195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd5d1737fe344f49857b25858415d6dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:00:57.916632 25967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd5d1737fe344f49857b25858415d6dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:00:57.928092 26339 tablet_bootstrap.cc:492] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: Bootstrap starting.
I20260809 05:00:57.930425 26341 tablet_bootstrap.cc:492] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: Bootstrap starting.
I20260809 05:00:57.931407 26342 tablet_bootstrap.cc:492] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: Bootstrap starting.
I20260809 05:00:57.933615 26339 tablet_bootstrap.cc:654] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: Neither blocks nor log segments found. Creating new log.
I20260809 05:00:57.934072 26341 tablet_bootstrap.cc:654] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: Neither blocks nor log segments found. Creating new log.
I20260809 05:00:57.934588 26342 tablet_bootstrap.cc:654] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: Neither blocks nor log segments found. Creating new log.
I20260809 05:00:57.934993 26339 log.cc:824] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: Log is configured to *not* fsync() on all Append() calls
I20260809 05:00:57.935231 26341 log.cc:824] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: Log is configured to *not* fsync() on all Append() calls
I20260809 05:00:57.935642 26342 log.cc:824] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: Log is configured to *not* fsync() on all Append() calls
I20260809 05:00:57.938021 26339 tablet_bootstrap.cc:492] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: No bootstrap required, opened a new log
I20260809 05:00:57.938277 26339 ts_tablet_manager.cc:1397] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 05:00:57.938664 26341 tablet_bootstrap.cc:492] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: No bootstrap required, opened a new log
I20260809 05:00:57.938663 26342 tablet_bootstrap.cc:492] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: No bootstrap required, opened a new log
I20260809 05:00:57.938930 26341 ts_tablet_manager.cc:1397] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 05:00:57.938930 26342 ts_tablet_manager.cc:1397] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 05:00:57.945999 26339 raft_consensus.cc:348] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71c28e285304820a5a85e1b5259161e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34587 } } peers { permanent_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38527 } } peers { permanent_uuid: "3694b7d1447041f88e62d03601a51dbc" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 34967 } }
I20260809 05:00:57.946388 26339 raft_consensus.cc:374] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:00:57.946516 26339 raft_consensus.cc:729] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a195f8fde441ab9a5ec2aa0f8b16fb, State: Initialized, Role: FOLLOWER
I20260809 05:00:57.947181 26339 consensus_queue.cc:260] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71c28e285304820a5a85e1b5259161e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34587 } } peers { permanent_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38527 } } peers { permanent_uuid: "3694b7d1447041f88e62d03601a51dbc" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 34967 } }
I20260809 05:00:57.948300 26303 heartbeater.cc:503] Master 127.29.243.254:43681 was elected leader, sending a full tablet report...
I20260809 05:00:57.948802 26339 ts_tablet_manager.cc:1428] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 05:00:57.949373 26341 raft_consensus.cc:348] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71c28e285304820a5a85e1b5259161e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34587 } } peers { permanent_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38527 } } peers { permanent_uuid: "3694b7d1447041f88e62d03601a51dbc" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 34967 } }
I20260809 05:00:57.949801 26341 raft_consensus.cc:374] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:00:57.949901 26341 raft_consensus.cc:729] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f71c28e285304820a5a85e1b5259161e, State: Initialized, Role: FOLLOWER
I20260809 05:00:57.950001 26342 raft_consensus.cc:348] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71c28e285304820a5a85e1b5259161e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34587 } } peers { permanent_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38527 } } peers { permanent_uuid: "3694b7d1447041f88e62d03601a51dbc" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 34967 } }
I20260809 05:00:57.950433 26342 raft_consensus.cc:374] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:00:57.950536 26342 raft_consensus.cc:729] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3694b7d1447041f88e62d03601a51dbc, State: Initialized, Role: FOLLOWER
I20260809 05:00:57.950628 26341 consensus_queue.cc:260] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71c28e285304820a5a85e1b5259161e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34587 } } peers { permanent_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38527 } } peers { permanent_uuid: "3694b7d1447041f88e62d03601a51dbc" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 34967 } }
I20260809 05:00:57.951599 26342 consensus_queue.cc:260] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71c28e285304820a5a85e1b5259161e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34587 } } peers { permanent_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38527 } } peers { permanent_uuid: "3694b7d1447041f88e62d03601a51dbc" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 34967 } }
I20260809 05:00:57.955989 26090 heartbeater.cc:503] Master 127.29.243.254:43681 was elected leader, sending a full tablet report...
I20260809 05:00:57.956733 26342 ts_tablet_manager.cc:1428] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: Time spent starting tablet: real 0.018s user 0.011s sys 0.006s
I20260809 05:00:57.958647 26341 ts_tablet_manager.cc:1428] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: Time spent starting tablet: real 0.020s user 0.012s sys 0.007s
W20260809 05:00:57.982496 25862 tablet.cc:2369] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:00:58.019251 26091 tablet.cc:2369] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:00:58.039798 26348 raft_consensus.cc:482] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:00:58.040089 26348 raft_consensus.cc:504] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71c28e285304820a5a85e1b5259161e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34587 } } peers { permanent_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38527 } } peers { permanent_uuid: "3694b7d1447041f88e62d03601a51dbc" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 34967 } }
I20260809 05:00:58.041991 26348 leader_election.cc:288] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f71c28e285304820a5a85e1b5259161e (127.29.243.193:34587), 3694b7d1447041f88e62d03601a51dbc (127.29.243.194:34967)
I20260809 05:00:58.048070 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:58.042385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:34587 (local address 127.29.243.195:40657)
0809 05:00:58.042631 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:58.042651 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:58.042796 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:58.043179 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:58.043183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:58.043214 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:00:58.043494 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:58.043509 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.044706 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.044711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:58.046349 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:58.046369 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.047528 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.047532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:58.047668 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:58.047694 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 05:00:58.047769 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 05:00:58.047860 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 05:00:58.048208 26330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:58.042401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:40657 (local address 127.29.243.193:34587)
0809 05:00:58.042586 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:58.042590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:58.042724 (+ 134us) server_negotiation.cc:407] Connection header received
0809 05:00:58.042980 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:58.042985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:58.043051 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:58.043137 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:00:58.043681 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.044582 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.046509 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.047359 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.047522 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:58.047598 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:58.047815 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 05:00:58.047958 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 05:00:58.049266 26352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:58.042845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:34967 (local address 127.29.243.195:56207)
0809 05:00:58.043748 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:58.043762 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:58.043856 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:58.044200 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:58.044203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:58.044228 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:00:58.044535 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:58.044543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.045594 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.045600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:58.047180 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:58.047192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.048691 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.048697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:58.048837 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:58.048861 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 05:00:58.048934 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 05:00:58.049024 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":582,"threads_started":1}
I20260809 05:00:58.049388 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:58.043167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:56207 (local address 127.29.243.194:34967)
0809 05:00:58.043408 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:58.043414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:58.043846 (+ 432us) server_negotiation.cc:407] Connection header received
0809 05:00:58.043970 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:58.043975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:58.044052 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:58.044150 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:00:58.044712 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.045449 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.047359 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.048497 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.048623 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:58.048673 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:58.049004 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 05:00:58.049150 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 05:00:58.049294 25791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd5d1737fe344f49857b25858415d6dc" candidate_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f71c28e285304820a5a85e1b5259161e" is_pre_election: true
I20260809 05:00:58.049980 25791 raft_consensus.cc:2454] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28a195f8fde441ab9a5ec2aa0f8b16fb in term 0.
I20260809 05:00:58.050259 26009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd5d1737fe344f49857b25858415d6dc" candidate_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3694b7d1447041f88e62d03601a51dbc" is_pre_election: true
I20260809 05:00:58.050642 26126 leader_election.cc:302] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a195f8fde441ab9a5ec2aa0f8b16fb, f71c28e285304820a5a85e1b5259161e; no voters:
I20260809 05:00:58.050760 26009 raft_consensus.cc:2454] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28a195f8fde441ab9a5ec2aa0f8b16fb in term 0.
I20260809 05:00:58.050999 26348 raft_consensus.cc:2781] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:00:58.051124 26348 raft_consensus.cc:482] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:00:58.051235 26348 raft_consensus.cc:3037] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:00:58.053721 26348 raft_consensus.cc:504] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71c28e285304820a5a85e1b5259161e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34587 } } peers { permanent_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38527 } } peers { permanent_uuid: "3694b7d1447041f88e62d03601a51dbc" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 34967 } }
I20260809 05:00:58.054354 26348 leader_election.cc:288] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [CANDIDATE]: Term 1 election: Requested vote from peers f71c28e285304820a5a85e1b5259161e (127.29.243.193:34587), 3694b7d1447041f88e62d03601a51dbc (127.29.243.194:34967)
I20260809 05:00:58.054924 26009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd5d1737fe344f49857b25858415d6dc" candidate_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3694b7d1447041f88e62d03601a51dbc"
I20260809 05:00:58.055119 26009 raft_consensus.cc:3037] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:00:58.057294 26009 raft_consensus.cc:2454] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28a195f8fde441ab9a5ec2aa0f8b16fb in term 1.
I20260809 05:00:58.057624 25791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd5d1737fe344f49857b25858415d6dc" candidate_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f71c28e285304820a5a85e1b5259161e"
I20260809 05:00:58.057691 26126 leader_election.cc:302] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a195f8fde441ab9a5ec2aa0f8b16fb, 3694b7d1447041f88e62d03601a51dbc; no voters:
I20260809 05:00:58.057794 25791 raft_consensus.cc:3037] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:00:58.057971 26348 raft_consensus.cc:2781] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:00:58.058513 26348 raft_consensus.cc:686] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 1 LEADER]: Becoming Leader. State: Replica: 28a195f8fde441ab9a5ec2aa0f8b16fb, State: Running, Role: LEADER
I20260809 05:00:58.059122 26348 consensus_queue.cc:237] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71c28e285304820a5a85e1b5259161e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34587 } } peers { permanent_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38527 } } peers { permanent_uuid: "3694b7d1447041f88e62d03601a51dbc" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 34967 } }
I20260809 05:00:58.060945 25791 raft_consensus.cc:2454] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28a195f8fde441ab9a5ec2aa0f8b16fb in term 1.
I20260809 05:00:58.064628 25536 catalog_manager.cc:5495] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 28a195f8fde441ab9a5ec2aa0f8b16fb (127.29.243.195). New cstate: current_term: 1 leader_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71c28e285304820a5a85e1b5259161e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28a195f8fde441ab9a5ec2aa0f8b16fb" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3694b7d1447041f88e62d03601a51dbc" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 34967 } health_report { overall_health: UNKNOWN } } }
W20260809 05:00:58.075500 26304 tablet.cc:2369] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:00:58.104321 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:58.098805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51604 (local address 127.29.243.195:38527)
0809 05:00:58.099040 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:58.099045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:58.099065 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:00:58.099343 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:58.099348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:58.099454 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:58.099594 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:00:58.099992 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.100721 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.102103 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.102507 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.102619 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:58.103168 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:00:58.103720 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:00:58.103860 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:58.103983 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 05:00:58.104071 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 05:00:58.113157 26009 raft_consensus.cc:1261] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc [term 1 FOLLOWER]: Refusing update from remote peer 28a195f8fde441ab9a5ec2aa0f8b16fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:00:58.113157 25791 raft_consensus.cc:1261] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e [term 1 FOLLOWER]: Refusing update from remote peer 28a195f8fde441ab9a5ec2aa0f8b16fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:00:58.114177 26353 consensus_queue.cc:1035] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3694b7d1447041f88e62d03601a51dbc" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:00:58.114540 26348 consensus_queue.cc:1035] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f71c28e285304820a5a85e1b5259161e" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:00:58.126087 26357 mvcc.cc:205] Tried to move back new op lower bound from 7316486791620403200 to 7316486791415099392. Current Snapshot: MvccSnapshot[applied={T|T < 7316486791620403200}]
I20260809 05:00:58.140095 25536 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:00:58.140348 25536 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:00:58.144150 25536 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0e095632f48474191d7a7873ea1801d: Sending DeleteTablet for 3 replicas of tablet cd5d1737fe344f49857b25858415d6dc
I20260809 05:00:58.145390 25749 tablet_service.cc:1514] Processing DeleteTablet for tablet cd5d1737fe344f49857b25858415d6dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:00:58 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:45198
I20260809 05:00:58.145753 26195 tablet_service.cc:1514] Processing DeleteTablet for tablet cd5d1737fe344f49857b25858415d6dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:00:58 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:51596
I20260809 05:00:58.145771 25967 tablet_service.cc:1514] Processing DeleteTablet for tablet cd5d1737fe344f49857b25858415d6dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:00:58 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:33888
I20260809 05:00:58.147926 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 25643
I20260809 05:00:58.155900 26373 tablet_replica.cc:330] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: stopping tablet replica
I20260809 05:00:58.156483 26373 raft_consensus.cc:2229] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:00:58.157066 26373 raft_consensus.cc:2258] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:00:58.157964 26370 tablet_replica.cc:330] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: stopping tablet replica
I20260809 05:00:58.158385 26370 raft_consensus.cc:2229] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:00:58.158893 26370 raft_consensus.cc:2258] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:00:58.159724 26372 tablet_replica.cc:330] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: stopping tablet replica
I20260809 05:00:58.160128 26372 raft_consensus.cc:2229] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:00:58.160845 26372 raft_consensus.cc:2258] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:00:58.162531 26370 ts_tablet_manager.cc:1905] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:00:58.162531 26373 ts_tablet_manager.cc:1905] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:00:58.163160 26372 ts_tablet_manager.cc:1905] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:00:58.169804 26370 ts_tablet_manager.cc:1918] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:00:58.170102 26370 log.cc:1196] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/cd5d1737fe344f49857b25858415d6dc
I20260809 05:00:58.170641 26372 ts_tablet_manager.cc:1918] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:00:58.170845 26370 ts_tablet_manager.cc:1939] T cd5d1737fe344f49857b25858415d6dc P f71c28e285304820a5a85e1b5259161e: Deleting consensus metadata
I20260809 05:00:58.170863 26372 log.cc:1196] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/cd5d1737fe344f49857b25858415d6dc
I20260809 05:00:58.171454 26372 ts_tablet_manager.cc:1939] T cd5d1737fe344f49857b25858415d6dc P 28a195f8fde441ab9a5ec2aa0f8b16fb: Deleting consensus metadata
I20260809 05:00:58.173206 25515 catalog_manager.cc:4841] TS f71c28e285304820a5a85e1b5259161e (127.29.243.193:34587): tablet cd5d1737fe344f49857b25858415d6dc (table test-table [id=76441bfd9a0b4f29a38db534c55e9a7d]) successfully deleted
I20260809 05:00:58.173751 25515 catalog_manager.cc:4841] TS 28a195f8fde441ab9a5ec2aa0f8b16fb (127.29.243.195:38527): tablet cd5d1737fe344f49857b25858415d6dc (table test-table [id=76441bfd9a0b4f29a38db534c55e9a7d]) successfully deleted
I20260809 05:00:58.177894 26373 ts_tablet_manager.cc:1918] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:00:58.178093 26373 log.cc:1196] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/cd5d1737fe344f49857b25858415d6dc
I20260809 05:00:58.178699 26373 ts_tablet_manager.cc:1939] T cd5d1737fe344f49857b25858415d6dc P 3694b7d1447041f88e62d03601a51dbc: Deleting consensus metadata
Aug 09 05:00:58 dist-test-slave-21b9 krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 05:00:58.180794 25515 catalog_manager.cc:4841] TS 3694b7d1447041f88e62d03601a51dbc (127.29.243.194:34967): tablet cd5d1737fe344f49857b25858415d6dc (table test-table [id=76441bfd9a0b4f29a38db534c55e9a7d]) successfully deleted
I20260809 05:00:58.186487 26330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:58.149504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45206 (local address 127.29.243.193:34587)
0809 05:00:58.149696 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:58.149703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:58.170394 (+ 20691us) server_negotiation.cc:407] Connection header received
0809 05:00:58.170952 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:58.170970 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:58.171069 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:58.171154 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:58.173226 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.174234 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.175676 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.176070 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.176190 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:58.180189 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:58.180201 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:58.180217 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:58.180257 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:58.183190 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:58.184157 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:58.184164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:58.184167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:58.184304 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:58.184810 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:58.184815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:58.184817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:58.185231 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:58.185494 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:58.186002 (+ 508us) server_negotiation.cc:299] Negotiation successful
0809 05:00:58.186178 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:00:58.187235 25855 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:00:58.283632 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 25643
I20260809 05:00:58.312070 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 25869
Aug 09 05:00:58 dist-test-slave-21b9 krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 05:00:58.327943 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:58.312958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33890 (local address 127.29.243.194:34967)
0809 05:00:58.313173 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:58.313178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:58.313199 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:00:58.313363 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:58.313369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:58.313458 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:58.313548 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:58.314896 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.315862 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.317334 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.317721 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.317839 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:58.322128 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:58.322144 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:58.322160 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:58.322197 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:58.325029 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:58.325682 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:58.325690 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:58.325693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:58.325825 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:58.326199 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:58.326203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:58.326205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:58.326710 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:58.326999 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:58.327371 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 05:00:58.327591 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 05:00:58.328552 26078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:00:58.416747 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 25869
I20260809 05:00:58.443787 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 26099
Aug 09 05:00:58 dist-test-slave-21b9 krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 05:00:58.462152 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:58.444645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51612 (local address 127.29.243.195:38527)
0809 05:00:58.444882 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:58.444889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:58.444913 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:00:58.445129 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:58.445136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:58.445225 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:58.445358 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:58.446876 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.447845 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.449393 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:58.450047 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:58.450204 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:58.455251 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:58.455289 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:58.455314 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:00:58.455359 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:58.458688 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:58.459694 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:58.459703 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:58.459705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:58.459909 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:00:58.460368 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:00:58.460373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:00:58.460375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:00:58.460784 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:58.461132 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:58.461477 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 05:00:58.461709 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260809 05:00:58.462978 26296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:00:58.587095 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 26099
I20260809 05:00:58.612624 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 25484
I20260809 05:00:58.613811 25590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:00:58.723413 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 25484
2026-08-09T05:00:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4648 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T05:00:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:00:58Z Disabled control of system clock
I20260809 05:00:58.823908 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:38413
--webserver_interface=127.29.243.254
--webserver_port=0
--builtin_ntp_servers=127.29.243.212:46727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:38413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:00:59.172858 26384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:59.173164 26384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:59.173280 26384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:59.180534 26384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:00:59.180639 26384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:59.180699 26384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:59.180773 26384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:00:59.180830 26384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:00:59.189982 26384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:46727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:38413
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:38413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:59.191344 26384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:59.193234 26384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:59.204798 26393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:59.204737 26397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:59.204768 26395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:59.205530 26384 server_base.cc:1034] running on GCE node
I20260809 05:00:59.206287 26384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:59.207670 26384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:59.208865 26384 hybrid_clock.cc:648] HybridClock initialized: now 1786251659208813 us; error 40 us; skew 500 ppm
I20260809 05:00:59.209332 26384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:00:59.211704 26384 webserver.cc:460] Webserver started at http://127.29.243.254:32785/ using document root <none> and password file <none>
I20260809 05:00:59.212356 26384 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:59.212455 26384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:59.212731 26384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:00:59.215693 26384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "e98eeab5afd24173bce1147758c29ae1"
format_stamp: "Formatted at 2026-08-09 05:00:59 on dist-test-slave-21b9"
server_key: "f5ed34aa51fecb3d529dbbdfcb1f975e"
server_key_iv: "12f6c0f1c90a9d2d94ecead65c038c13"
server_key_version: "encryptionkey@0"
I20260809 05:00:59.216468 26384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "e98eeab5afd24173bce1147758c29ae1"
format_stamp: "Formatted at 2026-08-09 05:00:59 on dist-test-slave-21b9"
server_key: "f5ed34aa51fecb3d529dbbdfcb1f975e"
server_key_iv: "12f6c0f1c90a9d2d94ecead65c038c13"
server_key_version: "encryptionkey@0"
I20260809 05:00:59.221766 26384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:00:59.225314 26403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:59.226696 26384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:00:59.226872 26384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "e98eeab5afd24173bce1147758c29ae1"
format_stamp: "Formatted at 2026-08-09 05:00:59 on dist-test-slave-21b9"
server_key: "f5ed34aa51fecb3d529dbbdfcb1f975e"
server_key_iv: "12f6c0f1c90a9d2d94ecead65c038c13"
server_key_version: "encryptionkey@0"
I20260809 05:00:59.227078 26384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:59.246001 26384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:59.246858 26384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:59.247131 26384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:59.264582 26384 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:38413
I20260809 05:00:59.264644 26481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:38413 every 8 connection(s)
I20260809 05:00:59.265916 26384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:00:59.269729 26482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:00:59.273375 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 26384
I20260809 05:00:59.277510 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:00:59.277978 30671 external_mini_cluster.cc:1428] Setting key dfc71e807bd4e11778b791f5e135bd74
I20260809 05:00:59.280220 26482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1: Bootstrap starting.
I20260809 05:00:59.284469 26482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1: Neither blocks nor log segments found. Creating new log.
I20260809 05:00:59.285830 26482 log.cc:824] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1: Log is configured to *not* fsync() on all Append() calls
I20260809 05:00:59.289026 26482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1: No bootstrap required, opened a new log
I20260809 05:00:59.293694 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:59.279888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41310 (local address 127.29.243.254:38413)
0809 05:00:59.280960 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:59.280975 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:59.281028 (+ 53us) server_negotiation.cc:407] Connection header received
0809 05:00:59.283444 (+ 2416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:59.283482 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:59.284074 (+ 592us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:59.284521 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:59.285091 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:59.286899 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:59.288469 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:59.289500 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:59.289706 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:59.290829 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:59.290839 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:59.290862 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:00:59.291617 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:59.291721 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:00:59.291751 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:59.291879 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:59.292122 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 05:00:59.292441 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":326,"threads_started":1}
I20260809 05:00:59.297720 26482 raft_consensus.cc:348] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e98eeab5afd24173bce1147758c29ae1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38413 } }
I20260809 05:00:59.298066 26482 raft_consensus.cc:374] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:00:59.298162 26482 raft_consensus.cc:729] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e98eeab5afd24173bce1147758c29ae1, State: Initialized, Role: FOLLOWER
I20260809 05:00:59.298739 26482 consensus_queue.cc:260] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e98eeab5afd24173bce1147758c29ae1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38413 } }
I20260809 05:00:59.298956 26482 raft_consensus.cc:388] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:00:59.299054 26482 raft_consensus.cc:482] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:00:59.299185 26482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:00:59.301265 26482 raft_consensus.cc:504] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e98eeab5afd24173bce1147758c29ae1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38413 } }
I20260809 05:00:59.301777 26482 leader_election.cc:302] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e98eeab5afd24173bce1147758c29ae1; no voters:
I20260809 05:00:59.302326 26482 leader_election.cc:288] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:00:59.302500 26487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:00:59.303249 26487 raft_consensus.cc:686] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [term 1 LEADER]: Becoming Leader. State: Replica: e98eeab5afd24173bce1147758c29ae1, State: Running, Role: LEADER
I20260809 05:00:59.303778 26487 consensus_queue.cc:237] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e98eeab5afd24173bce1147758c29ae1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38413 } }
I20260809 05:00:59.304304 26482 sys_catalog.cc:564] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:00:59.306871 26488 sys_catalog.cc:455] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e98eeab5afd24173bce1147758c29ae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e98eeab5afd24173bce1147758c29ae1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38413 } } }
I20260809 05:00:59.307399 26488 sys_catalog.cc:458] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [sys.catalog]: This master's current role is: LEADER
I20260809 05:00:59.307790 26490 sys_catalog.cc:455] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e98eeab5afd24173bce1147758c29ae1. Latest consensus state: current_term: 1 leader_uuid: "e98eeab5afd24173bce1147758c29ae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e98eeab5afd24173bce1147758c29ae1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38413 } } }
I20260809 05:00:59.308094 26490 sys_catalog.cc:458] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1 [sys.catalog]: This master's current role is: LEADER
I20260809 05:00:59.312317 26508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:00:59.316694 26508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:00:59.332921 26508 catalog_manager.cc:1347] Generated new cluster ID: 94546f8b8a9d4f319619184670507557
I20260809 05:00:59.333037 26508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:00:59.350646 26508 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:00:59.352131 26508 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:00:59.361181 26508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1: Generated new TSK 0
I20260809 05:00:59.361991 26508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:00:59.367237 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:38413
--builtin_ntp_servers=127.29.243.212:46727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:00:59.714742 26521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:00:59.715032 26521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:00:59.715166 26521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:00:59.722479 26521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:00:59.722587 26521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:00:59.722726 26521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 05:00:59.732381 26521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:46727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:38413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:00:59.733704 26521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:00:59.735689 26521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:00:59.748792 26527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:59.748863 26528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:00:59.748835 26531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:00:59.749795 26521 server_base.cc:1034] running on GCE node
I20260809 05:00:59.750480 26521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:00:59.751483 26521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:00:59.752691 26521 hybrid_clock.cc:648] HybridClock initialized: now 1786251659752641 us; error 45 us; skew 500 ppm
I20260809 05:00:59.753011 26521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:00:59.758375 26521 webserver.cc:460] Webserver started at http://127.29.243.193:46481/ using document root <none> and password file <none>
I20260809 05:00:59.759105 26521 fs_manager.cc:359] Metadata directory not provided
I20260809 05:00:59.759210 26521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:00:59.759550 26521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:00:59.762526 26521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "33b2cbd58dd8433b8313d61384a68cea"
format_stamp: "Formatted at 2026-08-09 05:00:59 on dist-test-slave-21b9"
server_key: "3d68045101a0c7fd99eeb7693058d3af"
server_key_iv: "e4210151389bb1f5e073a63c76a6658a"
server_key_version: "encryptionkey@0"
I20260809 05:00:59.763332 26521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "33b2cbd58dd8433b8313d61384a68cea"
format_stamp: "Formatted at 2026-08-09 05:00:59 on dist-test-slave-21b9"
server_key: "3d68045101a0c7fd99eeb7693058d3af"
server_key_iv: "e4210151389bb1f5e073a63c76a6658a"
server_key_version: "encryptionkey@0"
I20260809 05:00:59.768522 26521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 05:00:59.772177 26542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:59.773622 26521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:00:59.773803 26521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "33b2cbd58dd8433b8313d61384a68cea"
format_stamp: "Formatted at 2026-08-09 05:00:59 on dist-test-slave-21b9"
server_key: "3d68045101a0c7fd99eeb7693058d3af"
server_key_iv: "e4210151389bb1f5e073a63c76a6658a"
server_key_version: "encryptionkey@0"
I20260809 05:00:59.774051 26521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:00:59.790916 26521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:00:59.791823 26521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:00:59.792136 26521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:00:59.793354 26521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:00:59.794934 26521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:00:59.795048 26521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:59.795166 26521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:00:59.795243 26521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:00:59.845927 26521 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:37645
I20260809 05:00:59.846015 26725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:37645 every 8 connection(s)
I20260809 05:00:59.847594 26521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 05:00:59.848006 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 26521
I20260809 05:00:59.848218 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 05:00:59.848762 30671 external_mini_cluster.cc:1428] Setting key 17422e7b2b8aedd7b3c49d431a72f985
I20260809 05:00:59.851987 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:38413
--builtin_ntp_servers=127.29.243.212:46727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:00:59.878643 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:59.852071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:45047 (local address 127.29.243.254:38413)
0809 05:00:59.861736 (+ 9665us) server_negotiation.cc:206] Beginning negotiation
0809 05:00:59.861742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:00:59.867670 (+ 5928us) server_negotiation.cc:407] Connection header received
0809 05:00:59.869917 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:00:59.869923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:00:59.870024 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:00:59.870127 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:00:59.871755 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:59.872799 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:59.875122 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:00:59.875711 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:00:59.875825 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:59.877784 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:00:59.877789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:00:59.877793 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:00:59.877813 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:00:59.877853 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:00:59.877865 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:00:59.877956 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 05:00:59.878189 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 05:00:59.878273 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9526,"thread_start_us":9662,"threads_started":1}
I20260809 05:00:59.879971 26728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:00:59.851493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:38413 (local address 127.29.243.193:45047)
0809 05:00:59.867113 (+ 15620us) negotiation.cc:107] Waiting for socket to connect
0809 05:00:59.867177 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 05:00:59.869591 (+ 2414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:00:59.870559 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:00:59.870571 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:00:59.870693 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:00:59.871552 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:59.871580 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:59.873027 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:59.873032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:59.874850 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:00:59.874865 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:00:59.876244 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:00:59.876252 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:00:59.876535 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:00:59.877351 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:00:59.877355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:00:59.877411 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:00:59.877567 (+ 156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:00:59.877972 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:00:59.877983 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:00:59.877991 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 05:00:59.878178 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 05:00:59.878585 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14924,"thread_start_us":14685,"threads_started":1}
I20260809 05:00:59.881537 26726 heartbeater.cc:347] Connected to a master server at 127.29.243.254:38413
I20260809 05:00:59.881847 26726 heartbeater.cc:464] Registering TS with master...
I20260809 05:00:59.882678 26726 heartbeater.cc:511] Master 127.29.243.254:38413 requested a full tablet report, sending...
I20260809 05:00:59.885105 26428 ts_manager.cc:194] Registered new tserver with Master: 33b2cbd58dd8433b8313d61384a68cea (127.29.243.193:37645)
I20260809 05:00:59.887197 26428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.193:45047
W20260809 05:01:00.214452 26732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:00.214730 26732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:00.214840 26732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:00.222050 26732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:00.222157 26732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:00.222287 26732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 05:01:00.231490 26732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:46727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:38413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:00.232842 26732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:00.234640 26732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:00.247814 26739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:00.248029 26745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:00.247784 26738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:00.249199 26732 server_base.cc:1034] running on GCE node
I20260809 05:01:00.249779 26732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:00.250716 26732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:00.251943 26732 hybrid_clock.cc:648] HybridClock initialized: now 1786251660251877 us; error 49 us; skew 500 ppm
I20260809 05:01:00.252277 26732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:01:00.254583 26732 webserver.cc:460] Webserver started at http://127.29.243.194:38499/ using document root <none> and password file <none>
I20260809 05:01:00.255252 26732 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:00.255395 26732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:00.255700 26732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:00.258523 26732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "83ef337deaa54005a70c2222e26b6744"
format_stamp: "Formatted at 2026-08-09 05:01:00 on dist-test-slave-21b9"
server_key: "ba1b5076de781e2a2036f4e4b83ad75d"
server_key_iv: "b324aa1834fcebde6c16784e3eff8559"
server_key_version: "encryptionkey@0"
I20260809 05:01:00.259259 26732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "83ef337deaa54005a70c2222e26b6744"
format_stamp: "Formatted at 2026-08-09 05:01:00 on dist-test-slave-21b9"
server_key: "ba1b5076de781e2a2036f4e4b83ad75d"
server_key_iv: "b324aa1834fcebde6c16784e3eff8559"
server_key_version: "encryptionkey@0"
I20260809 05:01:00.264277 26732 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 05:01:00.267952 26752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:00.269409 26732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:01:00.269565 26732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "83ef337deaa54005a70c2222e26b6744"
format_stamp: "Formatted at 2026-08-09 05:01:00 on dist-test-slave-21b9"
server_key: "ba1b5076de781e2a2036f4e4b83ad75d"
server_key_iv: "b324aa1834fcebde6c16784e3eff8559"
server_key_version: "encryptionkey@0"
I20260809 05:01:00.269762 26732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:00.290572 26732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:00.291481 26732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:00.291774 26732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:00.292768 26732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:00.294358 26732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:00.294454 26732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:00.294564 26732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:00.294636 26732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:00.325712 26732 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:46461
I20260809 05:01:00.325788 26933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:46461 every 8 connection(s)
I20260809 05:01:00.327010 26732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 05:01:00.332459 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 26732
I20260809 05:01:00.332669 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 05:01:00.333212 30671 external_mini_cluster.cc:1428] Setting key 90317a5cf45234000a1cdece9210fd77
I20260809 05:01:00.336413 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:38413
--builtin_ntp_servers=127.29.243.212:46727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:01:00.340994 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.329709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:45449 (local address 127.29.243.254:38413)
0809 05:01:00.329903 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:00.329908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:00.331374 (+ 1466us) server_negotiation.cc:407] Connection header received
0809 05:01:00.333728 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:00.333732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:00.333811 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:00.333938 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:00.335060 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.335910 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.338242 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.338658 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.338767 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.340309 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:00.340314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:00.340318 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:00.340337 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:00.340369 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:00.340380 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:00.340449 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:00.340671 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 05:01:00.340749 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 05:01:00.342435 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.330189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:38413 (local address 127.29.243.194:45449)
0809 05:01:00.331191 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:00.331232 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:00.333421 (+ 2189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:00.334163 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:00.334175 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:00.334288 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:00.334905 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.334924 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.336094 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.336099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.338055 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.338073 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.338894 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.338901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.339177 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.339938 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:01:00.339941 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:00.340006 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:01:00.340119 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:00.340466 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:00.340476 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:00.340485 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 05:01:00.340664 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 05:01:00.341092 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":266,"threads_started":1}
I20260809 05:01:00.343924 26934 heartbeater.cc:347] Connected to a master server at 127.29.243.254:38413
I20260809 05:01:00.344308 26934 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:00.345240 26934 heartbeater.cc:511] Master 127.29.243.254:38413 requested a full tablet report, sending...
I20260809 05:01:00.346779 26428 ts_manager.cc:194] Registered new tserver with Master: 83ef337deaa54005a70c2222e26b6744 (127.29.243.194:46461)
I20260809 05:01:00.347625 26428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.194:45449
W20260809 05:01:00.684248 26939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:00.684531 26939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:00.684619 26939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:00.691718 26939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:00.691818 26939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:00.691921 26939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 05:01:00.700778 26939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:46727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:38413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:00.702032 26939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:00.703910 26939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:00.717077 26947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:00.717162 26949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:00.717130 26946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:00.717994 26939 server_base.cc:1034] running on GCE node
I20260809 05:01:00.718626 26939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:00.719616 26939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:00.720821 26939 hybrid_clock.cc:648] HybridClock initialized: now 1786251660720770 us; error 44 us; skew 500 ppm
I20260809 05:01:00.721180 26939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:01:00.723526 26939 webserver.cc:460] Webserver started at http://127.29.243.195:37159/ using document root <none> and password file <none>
I20260809 05:01:00.724261 26939 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:00.724366 26939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:00.724677 26939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:00.727607 26939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "747c7faa073d4997beceee583456d76c"
format_stamp: "Formatted at 2026-08-09 05:01:00 on dist-test-slave-21b9"
server_key: "d077bcaa8dc21c39fc887e50c7ef5ba5"
server_key_iv: "3d17b7ef5bc98a9350d54718bd981b49"
server_key_version: "encryptionkey@0"
I20260809 05:01:00.728379 26939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "747c7faa073d4997beceee583456d76c"
format_stamp: "Formatted at 2026-08-09 05:01:00 on dist-test-slave-21b9"
server_key: "d077bcaa8dc21c39fc887e50c7ef5ba5"
server_key_iv: "3d17b7ef5bc98a9350d54718bd981b49"
server_key_version: "encryptionkey@0"
I20260809 05:01:00.733601 26939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:01:00.737219 26955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:00.738634 26939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:01:00.738797 26939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "747c7faa073d4997beceee583456d76c"
format_stamp: "Formatted at 2026-08-09 05:01:00 on dist-test-slave-21b9"
server_key: "d077bcaa8dc21c39fc887e50c7ef5ba5"
server_key_iv: "3d17b7ef5bc98a9350d54718bd981b49"
server_key_version: "encryptionkey@0"
I20260809 05:01:00.739003 26939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:00.764197 26939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:00.765115 26939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:00.765419 26939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:00.766479 26939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:00.768028 26939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:00.768128 26939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:00.768235 26939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:00.768308 26939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:00.799633 26939 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:39701
I20260809 05:01:00.799695 27135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:39701 every 8 connection(s)
I20260809 05:01:00.801065 26939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 05:01:00.809347 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 26939
I20260809 05:01:00.809533 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 05:01:00.809984 30671 external_mini_cluster.cc:1428] Setting key fa5d9680a7e83613d6a2547aedc5718f
I20260809 05:01:00.814378 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.803973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:56873 (local address 127.29.243.254:38413)
0809 05:01:00.804170 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:00.804175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:00.805728 (+ 1553us) server_negotiation.cc:407] Connection header received
0809 05:01:00.807552 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:00.807557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:00.807644 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:00.807735 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:00.808871 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.809706 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.811547 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.812083 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.812195 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.813579 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:00.813585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:00.813588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:00.813605 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:00.813635 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:00.813647 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:00.813721 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:00.813942 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 05:01:00.814041 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:01:00.815436 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.804552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:38413 (local address 127.29.243.195:56873)
0809 05:01:00.805567 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:00.805613 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:00.807241 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:00.807976 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:00.807993 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:00.808089 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:00.808696 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.808713 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.809846 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.809851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.811393 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.811406 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.812308 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.812316 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.812547 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.813256 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:01:00.813259 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:00.813299 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:01:00.813386 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:00.813735 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:00.813747 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:00.813756 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 05:01:00.813914 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 05:01:00.814327 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":257,"threads_started":1}
I20260809 05:01:00.816774 27138 heartbeater.cc:347] Connected to a master server at 127.29.243.254:38413
I20260809 05:01:00.817101 27138 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:00.817867 27138 heartbeater.cc:511] Master 127.29.243.254:38413 requested a full tablet report, sending...
I20260809 05:01:00.819311 26428 ts_manager.cc:194] Registered new tserver with Master: 747c7faa073d4997beceee583456d76c (127.29.243.195:39701)
I20260809 05:01:00.820142 26428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.195:56873
I20260809 05:01:00.825253 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:01:00.835210 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.829336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41320 (local address 127.29.243.254:38413)
0809 05:01:00.829493 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:00.829498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:00.830000 (+ 502us) server_negotiation.cc:407] Connection header received
0809 05:01:00.830182 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:00.830186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:00.830259 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:00.830341 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:00.830870 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.831626 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.833027 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.833415 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.833522 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.834513 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:00.834519 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:00.834524 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:00.834545 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:00.834589 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:00.834602 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:00.834678 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:00.834812 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 05:01:00.834899 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 05:01:00.838723 26428 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:01:00.845690 26428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:01:00.877018 27157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.858980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:37645 (local address 127.0.0.1:56924)
0809 05:01:00.867963 (+ 8983us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:00.867982 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:00.868101 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:00.869015 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:00.869023 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:00.869052 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:00.869522 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.869535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.871831 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.871836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.874090 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.874104 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.875952 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.875957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.876135 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.876186 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 05:01:00.876317 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 05:01:00.876539 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8866,"mutex_wait_us":118,"thread_start_us":8739,"threads_started":1}
I20260809 05:01:00.877372 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.857128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:39701 (local address 127.0.0.1:34722)
0809 05:01:00.858017 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:00.858058 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:00.858267 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:00.870304 (+ 12037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:00.870308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:00.870333 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:00.870729 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.870741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.872523 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.872528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.874687 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.874700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.876496 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.876506 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.876703 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.876731 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 05:01:00.876798 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 05:01:00.876933 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":20,"thread_start_us":1066,"threads_started":1}
I20260809 05:01:00.877655 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.857679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59536 (local address 127.29.243.194:46461)
0809 05:01:00.868791 (+ 11112us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:00.868799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:00.868827 (+ 28us) server_negotiation.cc:407] Connection header received
0809 05:01:00.868927 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:00.868934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:00.869194 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:00.870038 (+ 844us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:00.870256 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.871750 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.874725 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.876145 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.876353 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.876585 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:00.876821 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 05:01:00.876986 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10963,"thread_start_us":10870,"threads_started":1}
I20260809 05:01:00.877911 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.857292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:46461 (local address 127.0.0.1:59536)
0809 05:01:00.858890 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:00.858905 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:00.859003 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:00.869409 (+ 10406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:00.869413 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:00.869442 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:00.869843 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.869856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.871900 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.871904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.874490 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.874505 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.876427 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.876434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.876626 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.876660 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 05:01:00.876751 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 05:01:00.876878 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1459,"mutex_wait_us":208,"thread_start_us":1308,"threads_started":1}
I20260809 05:01:00.878226 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.859457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.29.243.193:37645)
0809 05:01:00.868308 (+ 8851us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:00.868317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:00.868350 (+ 33us) server_negotiation.cc:407] Connection header received
0809 05:01:00.868456 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:00.868465 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:00.868752 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:00.869771 (+ 1019us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:00.870007 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.871676 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.874321 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.875775 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.875954 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.876203 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:00.877399 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0809 05:01:00.877685 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8656,"thread_start_us":8715,"threads_started":1}
I20260809 05:01:00.878321 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.857118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.29.243.195:39701)
0809 05:01:00.869699 (+ 12581us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:00.869707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:00.869743 (+ 36us) server_negotiation.cc:407] Connection header received
0809 05:01:00.869841 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:00.869848 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:00.870084 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:00.870893 (+ 809us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:00.871112 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.872382 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.874884 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.876284 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.877178 (+ 894us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.877707 (+ 529us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:00.877777 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 05:01:00.877974 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12419,"thread_start_us":12267,"threads_started":1}
I20260809 05:01:00.882050 27036 tablet_service.cc:1467] Processing CreateTablet for tablet 50e5168895a642609882b8b5d8048245 (DEFAULT_TABLE table=test-table [id=e8adcb09d03d450d9593404b6ab62710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:00.882093 26831 tablet_service.cc:1467] Processing CreateTablet for tablet 50e5168895a642609882b8b5d8048245 (DEFAULT_TABLE table=test-table [id=e8adcb09d03d450d9593404b6ab62710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:00.882503 26621 tablet_service.cc:1467] Processing CreateTablet for tablet 50e5168895a642609882b8b5d8048245 (DEFAULT_TABLE table=test-table [id=e8adcb09d03d450d9593404b6ab62710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:00.883433 27036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50e5168895a642609882b8b5d8048245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:00.883538 26831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50e5168895a642609882b8b5d8048245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:00.883790 26621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50e5168895a642609882b8b5d8048245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:00.890290 26726 heartbeater.cc:503] Master 127.29.243.254:38413 was elected leader, sending a full tablet report...
I20260809 05:01:00.895120 27160 tablet_bootstrap.cc:492] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c: Bootstrap starting.
I20260809 05:01:00.898909 27160 tablet_bootstrap.cc:654] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:00.899832 27161 tablet_bootstrap.cc:492] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744: Bootstrap starting.
I20260809 05:01:00.900257 27160 log.cc:824] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:00.900734 27162 tablet_bootstrap.cc:492] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea: Bootstrap starting.
I20260809 05:01:00.903030 27160 tablet_bootstrap.cc:492] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c: No bootstrap required, opened a new log
I20260809 05:01:00.903321 27160 ts_tablet_manager.cc:1397] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 05:01:00.904479 27162 tablet_bootstrap.cc:654] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:00.904522 27161 tablet_bootstrap.cc:654] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:00.905771 27161 log.cc:824] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:00.905745 27162 log.cc:824] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:00.908910 27161 tablet_bootstrap.cc:492] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744: No bootstrap required, opened a new log
I20260809 05:01:00.908982 27162 tablet_bootstrap.cc:492] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea: No bootstrap required, opened a new log
I20260809 05:01:00.909214 27161 ts_tablet_manager.cc:1397] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 05:01:00.909291 27162 ts_tablet_manager.cc:1397] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 05:01:00.912421 27160 raft_consensus.cc:348] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "747c7faa073d4997beceee583456d76c" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 39701 } } peers { permanent_uuid: "83ef337deaa54005a70c2222e26b6744" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46461 } } peers { permanent_uuid: "33b2cbd58dd8433b8313d61384a68cea" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37645 } }
I20260809 05:01:00.912882 27160 raft_consensus.cc:374] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:00.912989 27160 raft_consensus.cc:729] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 747c7faa073d4997beceee583456d76c, State: Initialized, Role: FOLLOWER
I20260809 05:01:00.913668 27160 consensus_queue.cc:260] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "747c7faa073d4997beceee583456d76c" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 39701 } } peers { permanent_uuid: "83ef337deaa54005a70c2222e26b6744" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46461 } } peers { permanent_uuid: "33b2cbd58dd8433b8313d61384a68cea" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37645 } }
I20260809 05:01:00.914788 27138 heartbeater.cc:503] Master 127.29.243.254:38413 was elected leader, sending a full tablet report...
I20260809 05:01:00.915470 27160 ts_tablet_manager.cc:1428] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260809 05:01:00.917519 27161 raft_consensus.cc:348] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "747c7faa073d4997beceee583456d76c" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 39701 } } peers { permanent_uuid: "83ef337deaa54005a70c2222e26b6744" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46461 } } peers { permanent_uuid: "33b2cbd58dd8433b8313d61384a68cea" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37645 } }
I20260809 05:01:00.917850 27161 raft_consensus.cc:374] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:00.917956 27161 raft_consensus.cc:729] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef337deaa54005a70c2222e26b6744, State: Initialized, Role: FOLLOWER
I20260809 05:01:00.918527 27161 consensus_queue.cc:260] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "747c7faa073d4997beceee583456d76c" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 39701 } } peers { permanent_uuid: "83ef337deaa54005a70c2222e26b6744" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46461 } } peers { permanent_uuid: "33b2cbd58dd8433b8313d61384a68cea" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37645 } }
I20260809 05:01:00.918660 27162 raft_consensus.cc:348] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "747c7faa073d4997beceee583456d76c" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 39701 } } peers { permanent_uuid: "83ef337deaa54005a70c2222e26b6744" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46461 } } peers { permanent_uuid: "33b2cbd58dd8433b8313d61384a68cea" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37645 } }
I20260809 05:01:00.919102 27162 raft_consensus.cc:374] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:00.919210 27162 raft_consensus.cc:729] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b2cbd58dd8433b8313d61384a68cea, State: Initialized, Role: FOLLOWER
I20260809 05:01:00.919868 27162 consensus_queue.cc:260] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "747c7faa073d4997beceee583456d76c" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 39701 } } peers { permanent_uuid: "83ef337deaa54005a70c2222e26b6744" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46461 } } peers { permanent_uuid: "33b2cbd58dd8433b8313d61384a68cea" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37645 } }
I20260809 05:01:00.920539 27161 ts_tablet_manager.cc:1428] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744: Time spent starting tablet: real 0.011s user 0.009s sys 0.003s
I20260809 05:01:00.920930 26934 heartbeater.cc:503] Master 127.29.243.254:38413 was elected leader, sending a full tablet report...
I20260809 05:01:00.921877 27162 ts_tablet_manager.cc:1428] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260809 05:01:00.969182 27175 raft_consensus.cc:482] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:01:00.969489 27175 raft_consensus.cc:504] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "747c7faa073d4997beceee583456d76c" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 39701 } } peers { permanent_uuid: "83ef337deaa54005a70c2222e26b6744" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46461 } } peers { permanent_uuid: "33b2cbd58dd8433b8313d61384a68cea" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37645 } }
I20260809 05:01:00.971182 27175 leader_election.cc:288] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 747c7faa073d4997beceee583456d76c (127.29.243.195:39701), 83ef337deaa54005a70c2222e26b6744 (127.29.243.194:46461)
I20260809 05:01:00.980024 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.971781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:34257 (local address 127.29.243.194:46461)
0809 05:01:00.971977 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:00.971983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:00.972635 (+ 652us) server_negotiation.cc:407] Connection header received
0809 05:01:00.972871 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:00.972876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:00.972999 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:00.973082 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:00.973811 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.974873 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.977870 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.979098 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.979257 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.979353 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:00.979629 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 05:01:00.979758 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 05:01:00.980033 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.971521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:46461 (local address 127.29.243.193:34257)
0809 05:01:00.972510 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:00.972531 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:00.972730 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:00.973125 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:00.973129 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:00.973183 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:00.973671 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.973683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.975035 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.975039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.977649 (+ 2610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.977665 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.979303 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.979312 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.979443 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.979481 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 05:01:00.979555 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 05:01:00.979649 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":139,"spinlock_wait_cycles":51712,"thread_start_us":629,"threads_started":1}
I20260809 05:01:00.980211 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.971770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:50879 (local address 127.29.243.195:39701)
0809 05:01:00.971980 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:00.971985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:00.972668 (+ 683us) server_negotiation.cc:407] Connection header received
0809 05:01:00.972828 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:00.972833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:00.972917 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:00.973016 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:00.973735 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.974907 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.977870 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.979117 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.979262 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.979353 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:00.979823 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 05:01:00.979964 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:01:00.980688 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:00.971662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:39701 (local address 127.29.243.193:50879)
0809 05:01:00.972570 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:00.972583 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:00.972693 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:00.973032 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:00.973035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:00.973060 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:00.973558 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.973572 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.975032 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.975038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.977649 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:00.977665 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:00.979490 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:00.979495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:00.979648 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:00.979680 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 05:01:00.979770 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 05:01:00.980357 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":115,"spinlock_wait_cycles":5504,"thread_start_us":537,"threads_started":1}
I20260809 05:01:00.981323 26862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50e5168895a642609882b8b5d8048245" candidate_uuid: "33b2cbd58dd8433b8313d61384a68cea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef337deaa54005a70c2222e26b6744" is_pre_election: true
I20260809 05:01:00.981746 27070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50e5168895a642609882b8b5d8048245" candidate_uuid: "33b2cbd58dd8433b8313d61384a68cea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "747c7faa073d4997beceee583456d76c" is_pre_election: true
I20260809 05:01:00.981865 26862 raft_consensus.cc:2454] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b2cbd58dd8433b8313d61384a68cea in term 0.
I20260809 05:01:00.982218 27070 raft_consensus.cc:2454] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33b2cbd58dd8433b8313d61384a68cea in term 0.
I20260809 05:01:00.982506 26546 leader_election.cc:302] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b2cbd58dd8433b8313d61384a68cea, 83ef337deaa54005a70c2222e26b6744; no voters:
I20260809 05:01:00.982921 27175 raft_consensus.cc:2781] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:01:00.983069 27175 raft_consensus.cc:482] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:01:00.983170 27175 raft_consensus.cc:3037] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:00.985363 27175 raft_consensus.cc:504] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "747c7faa073d4997beceee583456d76c" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 39701 } } peers { permanent_uuid: "83ef337deaa54005a70c2222e26b6744" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46461 } } peers { permanent_uuid: "33b2cbd58dd8433b8313d61384a68cea" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37645 } }
I20260809 05:01:00.985936 27175 leader_election.cc:288] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [CANDIDATE]: Term 1 election: Requested vote from peers 747c7faa073d4997beceee583456d76c (127.29.243.195:39701), 83ef337deaa54005a70c2222e26b6744 (127.29.243.194:46461)
I20260809 05:01:00.986395 26862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50e5168895a642609882b8b5d8048245" candidate_uuid: "33b2cbd58dd8433b8313d61384a68cea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef337deaa54005a70c2222e26b6744"
I20260809 05:01:00.986416 27070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50e5168895a642609882b8b5d8048245" candidate_uuid: "33b2cbd58dd8433b8313d61384a68cea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "747c7faa073d4997beceee583456d76c"
I20260809 05:01:00.986577 26862 raft_consensus.cc:3037] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:00.986580 27070 raft_consensus.cc:3037] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:00.988766 27070 raft_consensus.cc:2454] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b2cbd58dd8433b8313d61384a68cea in term 1.
I20260809 05:01:00.988780 26862 raft_consensus.cc:2454] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33b2cbd58dd8433b8313d61384a68cea in term 1.
I20260809 05:01:00.989137 26544 leader_election.cc:302] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33b2cbd58dd8433b8313d61384a68cea, 747c7faa073d4997beceee583456d76c; no voters:
I20260809 05:01:00.989403 27175 raft_consensus.cc:2781] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:00.989823 27175 raft_consensus.cc:686] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 1 LEADER]: Becoming Leader. State: Replica: 33b2cbd58dd8433b8313d61384a68cea, State: Running, Role: LEADER
I20260809 05:01:00.990270 27175 consensus_queue.cc:237] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "747c7faa073d4997beceee583456d76c" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 39701 } } peers { permanent_uuid: "83ef337deaa54005a70c2222e26b6744" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46461 } } peers { permanent_uuid: "33b2cbd58dd8433b8313d61384a68cea" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37645 } }
I20260809 05:01:00.994932 26428 catalog_manager.cc:5495] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea reported cstate change: term changed from 0 to 1, leader changed from <none> to 33b2cbd58dd8433b8313d61384a68cea (127.29.243.193). New cstate: current_term: 1 leader_uuid: "33b2cbd58dd8433b8313d61384a68cea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "747c7faa073d4997beceee583456d76c" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 39701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef337deaa54005a70c2222e26b6744" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33b2cbd58dd8433b8313d61384a68cea" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37645 } health_report { overall_health: HEALTHY } } }
I20260809 05:01:01.022845 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:01.017754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56928 (local address 127.29.243.193:37645)
0809 05:01:01.017951 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:01.017956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:01.017973 (+ 17us) server_negotiation.cc:407] Connection header received
0809 05:01:01.018067 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:01.018070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:01.018159 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:01.018286 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:01:01.018614 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:01.019358 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:01.020717 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:01.021143 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:01.021247 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:01.021784 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:01:01.022299 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:01:01.022437 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:01.022536 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 05:01:01.022603 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 05:01:01.031198 26862 raft_consensus.cc:1261] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744 [term 1 FOLLOWER]: Refusing update from remote peer 33b2cbd58dd8433b8313d61384a68cea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:01.031262 27070 raft_consensus.cc:1261] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c [term 1 FOLLOWER]: Refusing update from remote peer 33b2cbd58dd8433b8313d61384a68cea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:01.032203 27175 consensus_queue.cc:1035] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [LEADER]: Connected to new peer: Peer: permanent_uuid: "747c7faa073d4997beceee583456d76c" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 39701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:01:01.032501 27179 consensus_queue.cc:1035] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef337deaa54005a70c2222e26b6744" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 46461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:01:01.048138 27186 mvcc.cc:205] Tried to move back new op lower bound from 7316486803573874688 to 7316486803419676672. Current Snapshot: MvccSnapshot[applied={T|T < 7316486803573874688}]
I20260809 05:01:01.057718 26428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:01:01.057955 26428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:01:01.061532 26428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e98eeab5afd24173bce1147758c29ae1: Sending DeleteTablet for 3 replicas of tablet 50e5168895a642609882b8b5d8048245
I20260809 05:01:01.062669 27036 tablet_service.cc:1514] Processing DeleteTablet for tablet 50e5168895a642609882b8b5d8048245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:01 UTC) from {username='slave'} at 127.0.0.1:34722
I20260809 05:01:01.062779 26621 tablet_service.cc:1514] Processing DeleteTablet for tablet 50e5168895a642609882b8b5d8048245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:01 UTC) from {username='slave'} at 127.0.0.1:56924
I20260809 05:01:01.062806 26831 tablet_service.cc:1514] Processing DeleteTablet for tablet 50e5168895a642609882b8b5d8048245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:01 UTC) from {username='slave'} at 127.0.0.1:59536
I20260809 05:01:01.065533 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 26521
I20260809 05:01:01.075830 27202 tablet_replica.cc:330] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744: stopping tablet replica
I20260809 05:01:01.076491 27204 tablet_replica.cc:330] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea: stopping tablet replica
I20260809 05:01:01.077212 27204 raft_consensus.cc:2229] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:01:01.077785 27202 raft_consensus.cc:2229] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:01:01.078414 27202 raft_consensus.cc:2258] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:01.078537 27201 tablet_replica.cc:330] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c: stopping tablet replica
I20260809 05:01:01.078924 27201 raft_consensus.cc:2229] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:01:01.079475 27204 raft_consensus.cc:2258] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:01.079550 27201 raft_consensus.cc:2258] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:01.082041 27201 ts_tablet_manager.cc:1905] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:01.082459 27202 ts_tablet_manager.cc:1905] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:01.087811 27202 ts_tablet_manager.cc:1918] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:01.088012 27202 log.cc:1196] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/50e5168895a642609882b8b5d8048245
I20260809 05:01:01.088775 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:01.068031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56940 (local address 127.29.243.193:37645)
0809 05:01:01.068202 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:01.068208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:01.071134 (+ 2926us) server_negotiation.cc:407] Connection header received
0809 05:01:01.071363 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:01.071369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:01.071476 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:01.071596 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:01.076775 (+ 5179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:01.078040 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:01.080031 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:01.080671 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:01.080806 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:01.086721 (+ 5915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:01.086736 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:01.086754 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:01.087596 (+ 842us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:01.087664 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:01.087693 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:01.087794 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:01.088252 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 05:01:01.088372 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 05:01:01.089134 27201 ts_tablet_manager.cc:1918] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:01.089306 27201 log.cc:1196] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/50e5168895a642609882b8b5d8048245
I20260809 05:01:01.089854 27201 ts_tablet_manager.cc:1939] T 50e5168895a642609882b8b5d8048245 P 747c7faa073d4997beceee583456d76c: Deleting consensus metadata
I20260809 05:01:01.090468 26719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:01.091859 27202 ts_tablet_manager.cc:1939] T 50e5168895a642609882b8b5d8048245 P 83ef337deaa54005a70c2222e26b6744: Deleting consensus metadata
I20260809 05:01:01.091883 26406 catalog_manager.cc:4841] TS 747c7faa073d4997beceee583456d76c (127.29.243.195:39701): tablet 50e5168895a642609882b8b5d8048245 (table test-table [id=e8adcb09d03d450d9593404b6ab62710]) successfully deleted
I20260809 05:01:01.093775 26408 catalog_manager.cc:4841] TS 83ef337deaa54005a70c2222e26b6744 (127.29.243.194:46461): tablet 50e5168895a642609882b8b5d8048245 (table test-table [id=e8adcb09d03d450d9593404b6ab62710]) successfully deleted
I20260809 05:01:01.091437 27204 ts_tablet_manager.cc:1905] T 50e5168895a642609882b8b5d8048245 P 33b2cbd58dd8433b8313d61384a68cea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:01.188558 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 26521
W20260809 05:01:01.216104 26408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.243.193:37645 (error 108)
I20260809 05:01:01.216233 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 26732
I20260809 05:01:01.218104 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:01.217622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:37645 (local address 127.0.0.1:56948)
0809 05:01:01.217812 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:01.217933 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:37645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260809 05:01:01.218421 26408 catalog_manager.cc:4584] TS 33b2cbd58dd8433b8313d61384a68cea (127.29.243.193:37645): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50e5168895a642609882b8b5d8048245: Network error: Client connection negotiation failed: client connection to 127.29.243.193:37645: connect: Connection refused (error 111)
I20260809 05:01:01.223304 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:01.217104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.29.243.194:46461)
0809 05:01:01.217302 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:01.217308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:01.217335 (+ 27us) server_negotiation.cc:407] Connection header received
0809 05:01:01.217452 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:01.217458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:01.217563 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:01.217661 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:01.218111 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:01.219009 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:01.220328 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:01.220749 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:01.220878 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:01.221884 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:01.221893 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:01.221909 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:01.222588 (+ 679us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:01.222643 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:01.222667 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:01.222763 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:01.222884 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 05:01:01.222980 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:01:01.223944 26928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:01.272213 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:01.271623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:37645 (local address 127.0.0.1:56952)
0809 05:01:01.271858 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:01.271981 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:37645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 05:01:01.273356 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:01.272956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:37645 (local address 127.0.0.1:56968)
0809 05:01:01.273101 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:01.273183 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:37645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 05:01:01.316558 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 26732
I20260809 05:01:01.334125 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:01.333611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:37645 (local address 127.0.0.1:56980)
0809 05:01:01.333829 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:01.333947 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:37645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 05:01:01.335170 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:01.334784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:37645 (local address 127.0.0.1:56994)
0809 05:01:01.334964 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:01.335042 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:37645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 05:01:01.343350 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 26939
I20260809 05:01:01.350380 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:01.344140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34728 (local address 127.29.243.195:39701)
0809 05:01:01.344322 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:01.344327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:01.344349 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:01:01.344540 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:01.344544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:01.344648 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:01.344739 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:01.345186 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:01.345909 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:01.347243 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:01.347827 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:01.347946 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:01.349040 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:01.349050 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:01.349080 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:01.349663 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:01.349715 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:01.349740 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:01.349847 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:01.349979 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 05:01:01.350091 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:01:01.351038 27130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:01.416419 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:01.415842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:37645 (local address 127.0.0.1:57010)
0809 05:01:01.416095 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:01.416219 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:37645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 05:01:01.417641 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:01.417154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:37645 (local address 127.0.0.1:57016)
0809 05:01:01.417409 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:01.417501 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:37645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260809 05:01:01.439715 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 26939
I20260809 05:01:01.468000 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 26384
I20260809 05:01:01.469177 26474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:01.577888 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 26384
2026-08-09T05:01:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2854 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:01 dist-test-slave-21b9 krb5kdc[27222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:01 dist-test-slave-21b9 krb5kdc[27222](info): set up 2 sockets
Aug 09 05:01:01 dist-test-slave-21b9 krb5kdc[27222](info): commencing operation
krb5kdc: starting...
W20260809 05:01:02.834987 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:03 dist-test-slave-21b9 krb5kdc[27222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 05:01:03.148484 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:39203
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:37005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:39203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:03.492974 27247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:03.493276 27247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:03.493422 27247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:03.500921 27247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:03.501047 27247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:03.501127 27247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:03.501205 27247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:03.501267 27247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:03.511190 27247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:37005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:39203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:39203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:03.512607 27247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:03.513964 27247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:03.525645 27257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:03.525723 27255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:03.525744 27254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:03.526716 27247 server_base.cc:1034] running on GCE node
I20260809 05:01:03.527571 27247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:03.528963 27247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:03.530169 27247 hybrid_clock.cc:648] HybridClock initialized: now 1786251663530123 us; error 39 us; skew 500 ppm
I20260809 05:01:03.530510 27247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:03 dist-test-slave-21b9 krb5kdc[27222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251663, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:03.533655 27247 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:01:03.535334 27247 webserver.cc:460] Webserver started at http://127.29.243.254:32985/ using document root <none> and password file <none>
I20260809 05:01:03.536047 27247 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:03.536181 27247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:03.536535 27247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:03.539830 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "f42e6f28f0624244ad625245cc44a638"
format_stamp: "Formatted at 2026-08-09 05:01:03 on dist-test-slave-21b9"
server_key: "5931b78b07fc4ce14bcdb8f750e57d6b"
server_key_iv: "ef4266dceef2a964e1a0d83c5cc2fcce"
server_key_version: "encryptionkey@0"
I20260809 05:01:03.540627 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "f42e6f28f0624244ad625245cc44a638"
format_stamp: "Formatted at 2026-08-09 05:01:03 on dist-test-slave-21b9"
server_key: "5931b78b07fc4ce14bcdb8f750e57d6b"
server_key_iv: "ef4266dceef2a964e1a0d83c5cc2fcce"
server_key_version: "encryptionkey@0"
I20260809 05:01:03.546386 27247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:01:03.550499 27267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:03.551952 27247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:03.552129 27247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "f42e6f28f0624244ad625245cc44a638"
format_stamp: "Formatted at 2026-08-09 05:01:03 on dist-test-slave-21b9"
server_key: "5931b78b07fc4ce14bcdb8f750e57d6b"
server_key_iv: "ef4266dceef2a964e1a0d83c5cc2fcce"
server_key_version: "encryptionkey@0"
I20260809 05:01:03.552313 27247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:03.567590 27247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:03.571813 27247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:03.572090 27247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:03.589412 27247 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:39203
I20260809 05:01:03.589421 27339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:39203 every 8 connection(s)
I20260809 05:01:03.590791 27247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:03.594702 27340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:03.598145 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 27247
I20260809 05:01:03.598357 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:03.598969 30671 external_mini_cluster.cc:1428] Setting key 731b9da12dd666cb61e792dd7acf5741
I20260809 05:01:03.607645 27340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638: Bootstrap starting.
I20260809 05:01:03.611600 27340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:03.612918 27340 log.cc:824] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638: Log is configured to *not* fsync() on all Append() calls
Aug 09 05:01:03 dist-test-slave-21b9 krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:03.616264 27340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638: No bootstrap required, opened a new log
I20260809 05:01:03.623958 27345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:03.600999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47350 (local address 127.29.243.254:39203)
0809 05:01:03.601784 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:03.601796 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:03.601840 (+ 44us) server_negotiation.cc:407] Connection header received
0809 05:01:03.603579 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:03.603602 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:03.604006 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:03.604390 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:03.606162 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:03.607498 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:03.609490 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:03.610175 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:03.610303 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:03.615306 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:03.615319 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:03.615339 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:03.615394 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:03.619206 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:03.619958 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:03.619967 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:03.619976 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:03.620197 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:03.620529 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:03.620535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:03.620538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:03.621277 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:03.621583 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:03.621996 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 05:01:03.622461 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":335,"threads_started":1}
I20260809 05:01:03.626555 27340 raft_consensus.cc:348] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42e6f28f0624244ad625245cc44a638" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39203 } }
I20260809 05:01:03.627074 27340 raft_consensus.cc:374] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:03.627209 27340 raft_consensus.cc:729] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42e6f28f0624244ad625245cc44a638, State: Initialized, Role: FOLLOWER
I20260809 05:01:03.627802 27340 consensus_queue.cc:260] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42e6f28f0624244ad625245cc44a638" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39203 } }
I20260809 05:01:03.628031 27340 raft_consensus.cc:388] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:03.628165 27340 raft_consensus.cc:482] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:03.628312 27340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:03.630482 27340 raft_consensus.cc:504] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42e6f28f0624244ad625245cc44a638" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39203 } }
I20260809 05:01:03.630985 27340 leader_election.cc:302] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f42e6f28f0624244ad625245cc44a638; no voters:
I20260809 05:01:03.631578 27340 leader_election.cc:288] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:03.631727 27347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:03.632858 27347 raft_consensus.cc:686] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [term 1 LEADER]: Becoming Leader. State: Replica: f42e6f28f0624244ad625245cc44a638, State: Running, Role: LEADER
I20260809 05:01:03.633364 27347 consensus_queue.cc:237] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42e6f28f0624244ad625245cc44a638" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39203 } }
I20260809 05:01:03.633785 27340 sys_catalog.cc:564] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:03.637351 27348 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f42e6f28f0624244ad625245cc44a638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42e6f28f0624244ad625245cc44a638" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39203 } } }
I20260809 05:01:03.637776 27349 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f42e6f28f0624244ad625245cc44a638. Latest consensus state: current_term: 1 leader_uuid: "f42e6f28f0624244ad625245cc44a638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42e6f28f0624244ad625245cc44a638" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39203 } } }
I20260809 05:01:03.638260 27348 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:03.638432 27349 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:03.639382 27363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:03.643760 27363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:03.652200 27363 catalog_manager.cc:1347] Generated new cluster ID: 82fddfe5502241308b8665bd7de19a4c
I20260809 05:01:03.652303 27363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:03.664811 27363 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:03.666182 27363 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:03.677583 27363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638: Generated new TSK 0
I20260809 05:01:03.678417 27363 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 05:01:03.746418 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:39203
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:37005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:04.102830 27379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:04.103138 27379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:04.103286 27379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:04.110663 27379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:04.110781 27379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:04.110931 27379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 05:01:04.121018 27379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:37005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:39203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:04.122510 27379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:04.123926 27379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:04.137342 27386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:04.137441 27385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:04.137527 27389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:04.138547 27379 server_base.cc:1034] running on GCE node
I20260809 05:01:04.139237 27379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:04.140180 27379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:04.141426 27379 hybrid_clock.cc:648] HybridClock initialized: now 1786251664141380 us; error 36 us; skew 500 ppm
I20260809 05:01:04.141716 27379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:04 dist-test-slave-21b9 krb5kdc[27222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251664, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:04.144794 27379 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 05:01:04.146238 27379 webserver.cc:460] Webserver started at http://127.29.243.193:39297/ using document root <none> and password file <none>
I20260809 05:01:04.146920 27379 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:04.147038 27379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:04.147414 27379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:04.150326 27379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "611b112481ad4fcda7885b8bad05192f"
format_stamp: "Formatted at 2026-08-09 05:01:04 on dist-test-slave-21b9"
server_key: "970df64f446986b5e89955f4f6d43164"
server_key_iv: "a4f6cc96160c417510198dec70569df8"
server_key_version: "encryptionkey@0"
I20260809 05:01:04.151069 27379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "611b112481ad4fcda7885b8bad05192f"
format_stamp: "Formatted at 2026-08-09 05:01:04 on dist-test-slave-21b9"
server_key: "970df64f446986b5e89955f4f6d43164"
server_key_iv: "a4f6cc96160c417510198dec70569df8"
server_key_version: "encryptionkey@0"
I20260809 05:01:04.156723 27379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 05:01:04.160246 27399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:04.161561 27379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:01:04.161721 27379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "611b112481ad4fcda7885b8bad05192f"
format_stamp: "Formatted at 2026-08-09 05:01:04 on dist-test-slave-21b9"
server_key: "970df64f446986b5e89955f4f6d43164"
server_key_iv: "a4f6cc96160c417510198dec70569df8"
server_key_version: "encryptionkey@0"
I20260809 05:01:04.161917 27379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:04.185042 27379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:04.188932 27379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:04.189220 27379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:04.190193 27379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:04.191606 27379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:04.191692 27379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:04.191810 27379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:04.191864 27379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:04.223094 27379 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:41943
I20260809 05:01:04.223165 27582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:41943 every 8 connection(s)
I20260809 05:01:04.224545 27379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 05:01:04.227010 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 27379
I20260809 05:01:04.227226 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 05:01:04.227788 30671 external_mini_cluster.cc:1428] Setting key bd27dc656e43ac9fc2b37fdedcfe1b4e
Aug 09 05:01:04 dist-test-slave-21b9 krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251664, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:04.245589 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:04.227219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:50383 (local address 127.29.243.254:39203)
0809 05:01:04.227790 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:04.227796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:04.228884 (+ 1088us) server_negotiation.cc:407] Connection header received
0809 05:01:04.230836 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:04.230841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:04.230921 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:04.231043 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:04.232564 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:04.233542 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:04.235478 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:04.236006 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:04.236133 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:04.239938 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:04.239944 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:04.239947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:04.239980 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:04.242334 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:04.243180 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:04.243186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:04.243189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:04.243365 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:04.243774 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:04.243778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:04.243781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:04.244123 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:04.244267 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:04.245005 (+ 738us) server_negotiation.cc:299] Negotiation successful
0809 05:01:04.245227 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":281,"threads_started":1}
I20260809 05:01:04.246482 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:04.227818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:39203 (local address 127.29.243.193:50383)
0809 05:01:04.228730 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:04.228776 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:04.230547 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:04.231224 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:04.231232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:04.231821 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:04.232386 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:04.232410 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:04.233755 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:04.233760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:04.235330 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:04.235349 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:04.236241 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:04.236246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:04.236480 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:04.237144 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:04.237168 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:04.239674 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:04.242479 (+ 2805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:04.242488 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:04.242506 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:04.243058 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:04.243467 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:04.243471 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:04.243473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:04.243667 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:04.244251 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:04.244258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:04.244621 (+ 363us) client_negotiation.cc:769] Sending connection context
0809 05:01:04.244847 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 05:01:04.245246 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":326,"threads_started":1}
I20260809 05:01:04.248098 27583 heartbeater.cc:347] Connected to a master server at 127.29.243.254:39203
I20260809 05:01:04.248405 27583 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:04.249156 27583 heartbeater.cc:511] Master 127.29.243.254:39203 requested a full tablet report, sending...
I20260809 05:01:04.251529 27290 ts_manager.cc:194] Registered new tserver with Master: 611b112481ad4fcda7885b8bad05192f (127.29.243.193:41943)
I20260809 05:01:04.253474 27290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:50383
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 05:01:04.296342 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:39203
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:37005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:04.651532 27594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:04.651815 27594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:04.651952 27594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:04.659143 27594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:04.659252 27594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:04.659456 27594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 05:01:04.670017 27594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:37005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:39203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:04.671458 27594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:04.672861 27594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:04.686305 27605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:04.686337 27602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:04.686542 27601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:04.687055 27594 server_base.cc:1034] running on GCE node
I20260809 05:01:04.687758 27594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:04.688745 27594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:04.689967 27594 hybrid_clock.cc:648] HybridClock initialized: now 1786251664689901 us; error 54 us; skew 500 ppm
I20260809 05:01:04.690300 27594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:04 dist-test-slave-21b9 krb5kdc[27222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251664, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:04.693511 27594 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 05:01:04.695066 27594 webserver.cc:460] Webserver started at http://127.29.243.194:35621/ using document root <none> and password file <none>
I20260809 05:01:04.695832 27594 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:04.695959 27594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:04.696298 27594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:04.699455 27594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "cceef8f239094c1b960872b975cc7800"
format_stamp: "Formatted at 2026-08-09 05:01:04 on dist-test-slave-21b9"
server_key: "264c86be9044c9277bbb19936580ab31"
server_key_iv: "6252b121a9c3178073083f1561d0a9d9"
server_key_version: "encryptionkey@0"
I20260809 05:01:04.700237 27594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "cceef8f239094c1b960872b975cc7800"
format_stamp: "Formatted at 2026-08-09 05:01:04 on dist-test-slave-21b9"
server_key: "264c86be9044c9277bbb19936580ab31"
server_key_iv: "6252b121a9c3178073083f1561d0a9d9"
server_key_version: "encryptionkey@0"
I20260809 05:01:04.705742 27594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:01:04.709393 27620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:04.710879 27594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:01:04.711061 27594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "cceef8f239094c1b960872b975cc7800"
format_stamp: "Formatted at 2026-08-09 05:01:04 on dist-test-slave-21b9"
server_key: "264c86be9044c9277bbb19936580ab31"
server_key_iv: "6252b121a9c3178073083f1561d0a9d9"
server_key_version: "encryptionkey@0"
I20260809 05:01:04.711256 27594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:04.738442 27594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:04.742879 27594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:04.743180 27594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:04.744304 27594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:04.745787 27594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:04.745887 27594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:04.746021 27594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:04.746074 27594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:04.777190 27594 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:38205
I20260809 05:01:04.777242 27795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:38205 every 8 connection(s)
I20260809 05:01:04.778481 27594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 05:01:04.786298 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 27594
I20260809 05:01:04.786504 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 05:01:04.786945 30671 external_mini_cluster.cc:1428] Setting key 0c66ac94ba6ee30d519133b94faa811b
Aug 09 05:01:04 dist-test-slave-21b9 krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251664, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:04.798588 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:04.781143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:42445 (local address 127.29.243.254:39203)
0809 05:01:04.781586 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:04.781590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:04.782479 (+ 889us) server_negotiation.cc:407] Connection header received
0809 05:01:04.784435 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:04.784440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:04.784529 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:04.784626 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:04.786146 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:04.786929 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:04.788629 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:04.789056 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:04.789181 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:04.792868 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:04.792873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:04.792877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:04.792910 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:04.795438 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:04.796280 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:04.796283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:04.796286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:04.796421 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:04.796873 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:04.796876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:04.796878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:04.797137 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:04.797256 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:04.798107 (+ 851us) server_negotiation.cc:299] Negotiation successful
0809 05:01:04.798301 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":228,"threads_started":1}
I20260809 05:01:04.800019 27799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:04.781594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:39203 (local address 127.29.243.194:42445)
0809 05:01:04.782320 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:04.782359 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:04.784155 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:04.784876 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:04.784883 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:04.785393 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:04.785970 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:04.785988 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:04.787082 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:04.787086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:04.788479 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:04.788499 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:04.789288 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:04.789296 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:04.789512 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:04.790129 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:04.790152 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:04.792629 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:04.795571 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:04.795580 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:04.795596 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:04.796148 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:04.796526 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:04.796531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:04.796535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:04.796774 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:04.797260 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:04.797269 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:04.797691 (+ 422us) client_negotiation.cc:769] Sending connection context
0809 05:01:04.798004 (+ 313us) client_negotiation.cc:240] Negotiation successful
0809 05:01:04.798538 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":270,"threads_started":1}
I20260809 05:01:04.801569 27796 heartbeater.cc:347] Connected to a master server at 127.29.243.254:39203
I20260809 05:01:04.801872 27796 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:04.802663 27796 heartbeater.cc:511] Master 127.29.243.254:39203 requested a full tablet report, sending...
I20260809 05:01:04.804325 27290 ts_manager.cc:194] Registered new tserver with Master: cceef8f239094c1b960872b975cc7800 (127.29.243.194:38205)
I20260809 05:01:04.805128 27290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:42445
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 05:01:04.854212 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:39203
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:37005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:05.196838 27803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:05.197150 27803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:05.197291 27803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:05.204645 27803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:05.204771 27803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:05.204921 27803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 05:01:05.214823 27803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:37005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:39203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:05.216259 27803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:05.217607 27803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:05.231439 27819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:05.231462 27825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:05.231923 27817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:05.232163 27803 server_base.cc:1034] running on GCE node
I20260809 05:01:05.232770 27803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:05.233664 27803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:05.234864 27803 hybrid_clock.cc:648] HybridClock initialized: now 1786251665234813 us; error 46 us; skew 500 ppm
I20260809 05:01:05.235178 27803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:05 dist-test-slave-21b9 krb5kdc[27222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251665, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:05.238441 27803 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 05:01:05.239864 27803 webserver.cc:460] Webserver started at http://127.29.243.195:46539/ using document root <none> and password file <none>
I20260809 05:01:05.240530 27803 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:05.240644 27803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:05.240934 27803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:05.243948 27803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "fe820be709d14aa9926d231e62931115"
format_stamp: "Formatted at 2026-08-09 05:01:05 on dist-test-slave-21b9"
server_key: "82043f0a054c6728dd5b09c94d7e3307"
server_key_iv: "faa87df471cc1add3d592f39e23e2550"
server_key_version: "encryptionkey@0"
I20260809 05:01:05.244656 27803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "fe820be709d14aa9926d231e62931115"
format_stamp: "Formatted at 2026-08-09 05:01:05 on dist-test-slave-21b9"
server_key: "82043f0a054c6728dd5b09c94d7e3307"
server_key_iv: "faa87df471cc1add3d592f39e23e2550"
server_key_version: "encryptionkey@0"
I20260809 05:01:05.249689 27803 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:01:05.253211 27837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:05.254547 27803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:01:05.254693 27803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "fe820be709d14aa9926d231e62931115"
format_stamp: "Formatted at 2026-08-09 05:01:05 on dist-test-slave-21b9"
server_key: "82043f0a054c6728dd5b09c94d7e3307"
server_key_iv: "faa87df471cc1add3d592f39e23e2550"
server_key_version: "encryptionkey@0"
I20260809 05:01:05.254879 27803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:05.256562 27583 heartbeater.cc:503] Master 127.29.243.254:39203 was elected leader, sending a full tablet report...
I20260809 05:01:05.269356 27803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:05.273473 27803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:05.273756 27803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:05.274716 27803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:05.276153 27803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:05.276228 27803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:05.276340 27803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:05.276394 27803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:05.306245 27803 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:42507
I20260809 05:01:05.306277 28015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:42507 every 8 connection(s)
I20260809 05:01:05.307572 27803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 05:01:05.313825 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 27803
I20260809 05:01:05.314009 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 05:01:05.314482 30671 external_mini_cluster.cc:1428] Setting key a82e15202f664d02f77123e36754192d
Aug 09 05:01:05 dist-test-slave-21b9 krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251665, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:05.326989 28020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.310101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:46255 (local address 127.29.243.254:39203)
0809 05:01:05.310543 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:05.310547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:05.311458 (+ 911us) server_negotiation.cc:407] Connection header received
0809 05:01:05.313275 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:05.313280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:05.313375 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:05.313473 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:05.315135 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.316116 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.317619 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.318015 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.318124 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.321392 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:05.321397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:05.321400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:05.321432 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:05.324045 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:05.324725 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:05.324729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:05.324731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:05.324853 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:05.325267 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:05.325272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:05.325275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:05.325556 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:05.325698 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:05.326384 (+ 686us) server_negotiation.cc:299] Negotiation successful
0809 05:01:05.326619 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":212,"threads_started":1}
I20260809 05:01:05.328078 28022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.310568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:39203 (local address 127.29.243.195:46255)
0809 05:01:05.311302 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:05.311347 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:05.313005 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:05.313719 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:05.313727 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:05.314293 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:05.314891 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.314910 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.316268 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.316272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.317481 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.317495 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.318230 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.318238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.318456 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.319090 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:05.319113 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:05.321192 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:05.324191 (+ 2999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:05.324200 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:05.324217 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:05.324610 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:05.324957 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:05.324962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:05.324966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:05.325158 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:05.325684 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:05.325692 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:05.326030 (+ 338us) client_negotiation.cc:769] Sending connection context
0809 05:01:05.326271 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 05:01:05.326778 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":275,"threads_started":1}
I20260809 05:01:05.329470 28016 heartbeater.cc:347] Connected to a master server at 127.29.243.254:39203
I20260809 05:01:05.329774 28016 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:05.330587 28016 heartbeater.cc:511] Master 127.29.243.254:39203 requested a full tablet report, sending...
I20260809 05:01:05.332191 27289 ts_manager.cc:194] Registered new tserver with Master: fe820be709d14aa9926d231e62931115 (127.29.243.195:42507)
I20260809 05:01:05.333070 27289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:46255
I20260809 05:01:05.341539 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:01:05.357971 28020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.345458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47364 (local address 127.29.243.254:39203)
0809 05:01:05.345638 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:05.345644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:05.346084 (+ 440us) server_negotiation.cc:407] Connection header received
0809 05:01:05.346208 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:05.346212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:05.346272 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:05.346347 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:05.347750 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.348479 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.349762 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.350140 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.350228 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.352785 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:05.352790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:05.352794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:05.352824 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:05.355315 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:05.356108 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:05.356111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:05.356114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:05.356227 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:05.356640 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:05.356643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:05.356645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:05.356931 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:05.357164 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:05.357513 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 05:01:05.357658 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 05:01:05.361840 27289 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:01:05.368731 27289 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:01:05.389350 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.381099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45060 (local address 127.29.243.193:41943)
0809 05:01:05.381768 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:05.381777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:05.381809 (+ 32us) server_negotiation.cc:407] Connection header received
0809 05:01:05.381921 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:05.381930 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:05.382193 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:05.382418 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:05.383521 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.385040 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.386959 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.388162 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.388358 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.388557 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:05.388826 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 05:01:05.388987 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":310,"threads_started":1}
I20260809 05:01:05.389387 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.380751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:41943 (local address 127.0.0.1:45060)
0809 05:01:05.381553 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:05.381602 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:05.381750 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:05.382725 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:05.382732 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:05.382770 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:05.383299 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.383316 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.385212 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.385217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.386770 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.386786 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.388391 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.388397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.388603 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.388648 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 05:01:05.388757 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 05:01:05.388967 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":403,"threads_started":1}
I20260809 05:01:05.392220 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.381939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:42507 (local address 127.0.0.1:41798)
0809 05:01:05.382443 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:05.382461 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:05.382577 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:05.384423 (+ 1846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:05.384427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:05.384450 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:05.384952 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.384971 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.386942 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.386947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.389686 (+ 2739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.389700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.391447 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.391454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.391632 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.391659 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 05:01:05.391711 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 05:01:05.391799 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":183,"thread_start_us":259,"threads_started":1}
I20260809 05:01:05.392374 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.382547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:38205 (local address 127.0.0.1:57308)
0809 05:01:05.383154 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:05.383170 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:05.383302 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:05.384651 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:05.384656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:05.384682 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:05.385170 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.385183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.386945 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.386949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.389573 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.389590 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.391507 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.391517 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.391694 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.391726 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 05:01:05.391821 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 05:01:05.391947 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":59,"thread_start_us":629,"threads_started":1}
I20260809 05:01:05.392414 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.383035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57308 (local address 127.29.243.194:38205)
0809 05:01:05.384109 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:05.384117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:05.384141 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:01:05.384228 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:05.384234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:05.384446 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:05.384649 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:05.385332 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.386808 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.389749 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.391171 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.391361 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.391546 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:05.391880 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 05:01:05.392048 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":918,"thread_start_us":908,"threads_started":1}
I20260809 05:01:05.393476 27476 tablet_service.cc:1467] Processing CreateTablet for tablet f39f09548faf4787a181e134aa7cef06 (DEFAULT_TABLE table=test-table [id=a29ac8eda525442e95ab825527d896b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:05.394263 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.382256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.29.243.195:42507)
0809 05:01:05.383833 (+ 1577us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:05.383842 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:05.383875 (+ 33us) server_negotiation.cc:407] Connection header received
0809 05:01:05.383985 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:05.383993 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:05.384215 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:05.384923 (+ 708us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:05.385196 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.386785 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.389846 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.391191 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.393523 (+ 2332us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.393692 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:05.393765 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 05:01:05.393917 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1420,"thread_start_us":1274,"threads_started":1}
I20260809 05:01:05.394927 27476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f39f09548faf4787a181e134aa7cef06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:05.395910 27698 tablet_service.cc:1467] Processing CreateTablet for tablet f39f09548faf4787a181e134aa7cef06 (DEFAULT_TABLE table=test-table [id=a29ac8eda525442e95ab825527d896b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:05.397017 27698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f39f09548faf4787a181e134aa7cef06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:05.402176 27909 tablet_service.cc:1467] Processing CreateTablet for tablet f39f09548faf4787a181e134aa7cef06 (DEFAULT_TABLE table=test-table [id=a29ac8eda525442e95ab825527d896b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:05.403260 27909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f39f09548faf4787a181e134aa7cef06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:05.407922 28045 tablet_bootstrap.cc:492] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: Bootstrap starting.
I20260809 05:01:05.410815 28046 tablet_bootstrap.cc:492] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: Bootstrap starting.
I20260809 05:01:05.411669 28045 tablet_bootstrap.cc:654] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:05.412945 28045 log.cc:824] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:05.413902 28046 tablet_bootstrap.cc:654] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:05.415109 28046 log.cc:824] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:05.417064 28045 tablet_bootstrap.cc:492] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: No bootstrap required, opened a new log
I20260809 05:01:05.417309 28045 ts_tablet_manager.cc:1397] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260809 05:01:05.417788 28046 tablet_bootstrap.cc:492] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: No bootstrap required, opened a new log
I20260809 05:01:05.418016 28046 ts_tablet_manager.cc:1397] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260809 05:01:05.419464 28053 tablet_bootstrap.cc:492] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: Bootstrap starting.
I20260809 05:01:05.422793 28053 tablet_bootstrap.cc:654] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:05.424039 28053 log.cc:824] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:05.424702 28045 raft_consensus.cc:348] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611b112481ad4fcda7885b8bad05192f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41943 } } peers { permanent_uuid: "fe820be709d14aa9926d231e62931115" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 42507 } } peers { permanent_uuid: "cceef8f239094c1b960872b975cc7800" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38205 } }
I20260809 05:01:05.425199 28045 raft_consensus.cc:374] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:05.425307 28045 raft_consensus.cc:729] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611b112481ad4fcda7885b8bad05192f, State: Initialized, Role: FOLLOWER
I20260809 05:01:05.425966 28045 consensus_queue.cc:260] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611b112481ad4fcda7885b8bad05192f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41943 } } peers { permanent_uuid: "fe820be709d14aa9926d231e62931115" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 42507 } } peers { permanent_uuid: "cceef8f239094c1b960872b975cc7800" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38205 } }
I20260809 05:01:05.426980 28046 raft_consensus.cc:348] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611b112481ad4fcda7885b8bad05192f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41943 } } peers { permanent_uuid: "fe820be709d14aa9926d231e62931115" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 42507 } } peers { permanent_uuid: "cceef8f239094c1b960872b975cc7800" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38205 } }
I20260809 05:01:05.427425 28046 raft_consensus.cc:374] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:05.427534 28046 raft_consensus.cc:729] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cceef8f239094c1b960872b975cc7800, State: Initialized, Role: FOLLOWER
I20260809 05:01:05.427713 28045 ts_tablet_manager.cc:1428] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260809 05:01:05.427983 28053 tablet_bootstrap.cc:492] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: No bootstrap required, opened a new log
I20260809 05:01:05.428248 28053 ts_tablet_manager.cc:1397] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260809 05:01:05.428197 28046 consensus_queue.cc:260] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611b112481ad4fcda7885b8bad05192f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41943 } } peers { permanent_uuid: "fe820be709d14aa9926d231e62931115" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 42507 } } peers { permanent_uuid: "cceef8f239094c1b960872b975cc7800" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38205 } }
I20260809 05:01:05.429288 27796 heartbeater.cc:503] Master 127.29.243.254:39203 was elected leader, sending a full tablet report...
I20260809 05:01:05.429975 28046 ts_tablet_manager.cc:1428] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260809 05:01:05.436882 28053 raft_consensus.cc:348] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611b112481ad4fcda7885b8bad05192f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41943 } } peers { permanent_uuid: "fe820be709d14aa9926d231e62931115" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 42507 } } peers { permanent_uuid: "cceef8f239094c1b960872b975cc7800" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38205 } }
I20260809 05:01:05.437240 28053 raft_consensus.cc:374] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:05.437338 28053 raft_consensus.cc:729] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe820be709d14aa9926d231e62931115, State: Initialized, Role: FOLLOWER
I20260809 05:01:05.437850 28053 consensus_queue.cc:260] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611b112481ad4fcda7885b8bad05192f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41943 } } peers { permanent_uuid: "fe820be709d14aa9926d231e62931115" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 42507 } } peers { permanent_uuid: "cceef8f239094c1b960872b975cc7800" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38205 } }
I20260809 05:01:05.438848 28016 heartbeater.cc:503] Master 127.29.243.254:39203 was elected leader, sending a full tablet report...
I20260809 05:01:05.439457 28053 ts_tablet_manager.cc:1428] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
W20260809 05:01:05.476913 27585 tablet.cc:2369] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:01:05.530571 27797 tablet.cc:2369] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:01:05.559171 28018 tablet.cc:2369] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:01:05.597721 28058 raft_consensus.cc:482] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:01:05.598055 28058 raft_consensus.cc:504] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611b112481ad4fcda7885b8bad05192f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41943 } } peers { permanent_uuid: "fe820be709d14aa9926d231e62931115" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 42507 } } peers { permanent_uuid: "cceef8f239094c1b960872b975cc7800" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38205 } }
I20260809 05:01:05.599891 28058 leader_election.cc:288] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611b112481ad4fcda7885b8bad05192f (127.29.243.193:41943), fe820be709d14aa9926d231e62931115 (127.29.243.195:42507)
I20260809 05:01:05.607103 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.600349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:48947 (local address 127.29.243.193:41943)
0809 05:01:05.600603 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:05.600610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:05.601197 (+ 587us) server_negotiation.cc:407] Connection header received
0809 05:01:05.601505 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:05.601511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:05.601593 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:05.601697 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:05.602428 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.603337 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.605279 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.606134 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.606252 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.606307 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:05.606718 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 05:01:05.606848 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 05:01:05.607170 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.600258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:41943 (local address 127.29.243.194:48947)
0809 05:01:05.601099 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:05.601117 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:05.601326 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:05.601803 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:05.601809 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:05.601847 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:05.602267 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.602282 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.603466 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.603471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.605116 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.605129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.606350 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.606356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.606542 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.606580 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 05:01:05.606665 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 05:01:05.606757 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":22,"spinlock_wait_cycles":35072,"thread_start_us":522,"threads_started":1}
I20260809 05:01:05.607251 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.601501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:47567 (local address 127.29.243.195:42507)
0809 05:01:05.601703 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:05.601708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:05.601996 (+ 288us) server_negotiation.cc:407] Connection header received
0809 05:01:05.602128 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:05.602134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:05.602214 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:05.602304 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:05.602887 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.603767 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.605564 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.606353 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.606500 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.606569 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:05.606895 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 05:01:05.607028 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:01:05.607399 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.601263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:42507 (local address 127.29.243.194:47567)
0809 05:01:05.601881 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:05.601898 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:05.602016 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:05.602332 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:05.602335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:05.602358 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:05.602742 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.602755 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.603898 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.603902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.605410 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:05.605425 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.606566 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.606571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:05.606727 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.606760 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 05:01:05.606847 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 05:01:05.606968 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":33,"thread_start_us":335,"threads_started":1}
I20260809 05:01:05.608279 27509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f39f09548faf4787a181e134aa7cef06" candidate_uuid: "cceef8f239094c1b960872b975cc7800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611b112481ad4fcda7885b8bad05192f" is_pre_election: true
I20260809 05:01:05.608595 27935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f39f09548faf4787a181e134aa7cef06" candidate_uuid: "cceef8f239094c1b960872b975cc7800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe820be709d14aa9926d231e62931115" is_pre_election: true
I20260809 05:01:05.608764 27509 raft_consensus.cc:2454] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cceef8f239094c1b960872b975cc7800 in term 0.
I20260809 05:01:05.609059 27935 raft_consensus.cc:2454] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cceef8f239094c1b960872b975cc7800 in term 0.
I20260809 05:01:05.609440 27630 leader_election.cc:302] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611b112481ad4fcda7885b8bad05192f, cceef8f239094c1b960872b975cc7800; no voters:
I20260809 05:01:05.609831 28058 raft_consensus.cc:2781] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:01:05.609961 28058 raft_consensus.cc:482] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:01:05.610221 28058 raft_consensus.cc:3037] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:05.612504 28058 raft_consensus.cc:504] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611b112481ad4fcda7885b8bad05192f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41943 } } peers { permanent_uuid: "fe820be709d14aa9926d231e62931115" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 42507 } } peers { permanent_uuid: "cceef8f239094c1b960872b975cc7800" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38205 } }
I20260809 05:01:05.613050 28058 leader_election.cc:288] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [CANDIDATE]: Term 1 election: Requested vote from peers 611b112481ad4fcda7885b8bad05192f (127.29.243.193:41943), fe820be709d14aa9926d231e62931115 (127.29.243.195:42507)
I20260809 05:01:05.613476 27509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f39f09548faf4787a181e134aa7cef06" candidate_uuid: "cceef8f239094c1b960872b975cc7800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611b112481ad4fcda7885b8bad05192f"
I20260809 05:01:05.613590 27935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f39f09548faf4787a181e134aa7cef06" candidate_uuid: "cceef8f239094c1b960872b975cc7800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe820be709d14aa9926d231e62931115"
I20260809 05:01:05.613685 27509 raft_consensus.cc:3037] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:05.613782 27935 raft_consensus.cc:3037] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:05.616850 27935 raft_consensus.cc:2454] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cceef8f239094c1b960872b975cc7800 in term 1.
I20260809 05:01:05.616859 27509 raft_consensus.cc:2454] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cceef8f239094c1b960872b975cc7800 in term 1.
I20260809 05:01:05.617238 27630 leader_election.cc:302] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cceef8f239094c1b960872b975cc7800, fe820be709d14aa9926d231e62931115; no voters:
I20260809 05:01:05.617513 28058 raft_consensus.cc:2781] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:05.617924 28058 raft_consensus.cc:686] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 1 LEADER]: Becoming Leader. State: Replica: cceef8f239094c1b960872b975cc7800, State: Running, Role: LEADER
I20260809 05:01:05.618701 28058 consensus_queue.cc:237] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611b112481ad4fcda7885b8bad05192f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41943 } } peers { permanent_uuid: "fe820be709d14aa9926d231e62931115" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 42507 } } peers { permanent_uuid: "cceef8f239094c1b960872b975cc7800" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38205 } }
I20260809 05:01:05.623145 27289 catalog_manager.cc:5495] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 reported cstate change: term changed from 0 to 1, leader changed from <none> to cceef8f239094c1b960872b975cc7800 (127.29.243.194). New cstate: current_term: 1 leader_uuid: "cceef8f239094c1b960872b975cc7800" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611b112481ad4fcda7885b8bad05192f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe820be709d14aa9926d231e62931115" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 42507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cceef8f239094c1b960872b975cc7800" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38205 } health_report { overall_health: HEALTHY } } }
I20260809 05:01:05.688859 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.683635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57320 (local address 127.29.243.194:38205)
0809 05:01:05.683874 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:05.683879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:05.683899 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:01:05.684060 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:05.684064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:05.684145 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:05.684269 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:01:05.684616 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.685367 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.686747 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.687157 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.687246 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.687765 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:01:05.688260 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:01:05.688440 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:05.688513 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 05:01:05.688565 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 05:01:05.696981 27935 raft_consensus.cc:1261] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115 [term 1 FOLLOWER]: Refusing update from remote peer cceef8f239094c1b960872b975cc7800: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:05.696987 27509 raft_consensus.cc:1261] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f [term 1 FOLLOWER]: Refusing update from remote peer cceef8f239094c1b960872b975cc7800: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:05.697908 28062 consensus_queue.cc:1035] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe820be709d14aa9926d231e62931115" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 42507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:01:05.698241 28058 consensus_queue.cc:1035] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [LEADER]: Connected to new peer: Peer: permanent_uuid: "611b112481ad4fcda7885b8bad05192f" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 41943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:01:05.724053 27290 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:01:05.724321 27290 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:01:05.727733 27290 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f42e6f28f0624244ad625245cc44a638: Sending DeleteTablet for 3 replicas of tablet f39f09548faf4787a181e134aa7cef06
I20260809 05:01:05.728856 27476 tablet_service.cc:1514] Processing DeleteTablet for tablet f39f09548faf4787a181e134aa7cef06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:05 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:45060
I20260809 05:01:05.728991 27909 tablet_service.cc:1514] Processing DeleteTablet for tablet f39f09548faf4787a181e134aa7cef06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:05 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:41798
I20260809 05:01:05.729142 27698 tablet_service.cc:1514] Processing DeleteTablet for tablet f39f09548faf4787a181e134aa7cef06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:05 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:57308
I20260809 05:01:05.731565 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 27379
I20260809 05:01:05.740077 28084 tablet_replica.cc:330] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: stopping tablet replica
I20260809 05:01:05.740556 28084 raft_consensus.cc:2229] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:01:05.741318 28084 raft_consensus.cc:2258] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:05.743827 28083 tablet_replica.cc:330] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: stopping tablet replica
I20260809 05:01:05.744151 28082 tablet_replica.cc:330] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: stopping tablet replica
I20260809 05:01:05.744251 28083 raft_consensus.cc:2229] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:01:05.744541 28082 raft_consensus.cc:2229] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:01:05.744993 28083 raft_consensus.cc:2258] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:05.744993 28082 raft_consensus.cc:2258] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:05.747244 28082 ts_tablet_manager.cc:1905] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:05.747244 28083 ts_tablet_manager.cc:1905] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:05.747949 28084 ts_tablet_manager.cc:1905] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:05.754196 28083 ts_tablet_manager.cc:1918] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:05.754196 28082 ts_tablet_manager.cc:1918] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:05.754361 28083 log.cc:1196] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/f39f09548faf4787a181e134aa7cef06
I20260809 05:01:05.754374 28082 log.cc:1196] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/f39f09548faf4787a181e134aa7cef06
I20260809 05:01:05.754870 28082 ts_tablet_manager.cc:1939] T f39f09548faf4787a181e134aa7cef06 P 611b112481ad4fcda7885b8bad05192f: Deleting consensus metadata
I20260809 05:01:05.754881 28083 ts_tablet_manager.cc:1939] T f39f09548faf4787a181e134aa7cef06 P fe820be709d14aa9926d231e62931115: Deleting consensus metadata
I20260809 05:01:05.755843 28084 ts_tablet_manager.cc:1918] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:05.756098 28084 log.cc:1196] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/f39f09548faf4787a181e134aa7cef06
I20260809 05:01:05.756866 27271 catalog_manager.cc:4841] TS 611b112481ad4fcda7885b8bad05192f (127.29.243.193:41943): tablet f39f09548faf4787a181e134aa7cef06 (table test-table [id=a29ac8eda525442e95ab825527d896b7]) successfully deleted
I20260809 05:01:05.757225 27271 catalog_manager.cc:4841] TS fe820be709d14aa9926d231e62931115 (127.29.243.195:42507): tablet f39f09548faf4787a181e134aa7cef06 (table test-table [id=a29ac8eda525442e95ab825527d896b7]) successfully deleted
I20260809 05:01:05.757364 28084 ts_tablet_manager.cc:1939] T f39f09548faf4787a181e134aa7cef06 P cceef8f239094c1b960872b975cc7800: Deleting consensus metadata
I20260809 05:01:05.759238 27271 catalog_manager.cc:4841] TS cceef8f239094c1b960872b975cc7800 (127.29.243.194:38205): tablet f39f09548faf4787a181e134aa7cef06 (table test-table [id=a29ac8eda525442e95ab825527d896b7]) successfully deleted
Aug 09 05:01:05 dist-test-slave-21b9 krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 05:01:05.772908 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.733072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45072 (local address 127.29.243.193:41943)
0809 05:01:05.733253 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:05.733260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:05.745933 (+ 12673us) server_negotiation.cc:407] Connection header received
0809 05:01:05.756922 (+ 10989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:05.756928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:05.757025 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:05.757122 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:05.759030 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.760244 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.761556 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.762189 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.762325 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.767050 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:05.767062 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:05.767079 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:05.767111 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:05.770005 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:05.770864 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:05.770872 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:05.770875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:05.771011 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:05.771497 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:05.771502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:05.771504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:05.771878 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:05.772119 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:05.772465 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 05:01:05.772629 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 05:01:05.773590 27575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:05.876358 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 27379
I20260809 05:01:05.912128 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 27594
Aug 09 05:01:05 dist-test-slave-21b9 krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 05:01:05.929695 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:05.913067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57322 (local address 127.29.243.194:38205)
0809 05:01:05.913226 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:05.913230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:05.913248 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:01:05.913369 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:05.913372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:05.913437 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:05.913517 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:05.915191 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.916154 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.917987 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:05.918446 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:05.918545 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:05.923705 (+ 5160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:05.923718 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:05.923738 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:05.923777 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:05.926794 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:05.927545 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:05.927553 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:05.927555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:05.927706 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:05.928079 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:05.928084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:05.928086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:05.928460 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:05.928785 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:05.929045 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 05:01:05.929275 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 05:01:05.930469 27789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:06.046116 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 27594
I20260809 05:01:06.077214 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 27803
Aug 09 05:01:06 dist-test-slave-21b9 krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 05:01:06.095106 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:06.077975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.29.243.195:42507)
0809 05:01:06.078136 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:06.078141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:06.078219 (+ 78us) server_negotiation.cc:407] Connection header received
0809 05:01:06.078368 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:06.078372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:06.078457 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:06.078590 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:06.079996 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:06.080704 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:06.081910 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:06.082269 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:06.082358 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:06.086821 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:06.086837 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:06.086857 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:06.086897 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:06.092547 (+ 5650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:06.093244 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:06.093253 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:06.093256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:06.093463 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:06.093792 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:06.093795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:06.093797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:06.094160 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:06.094398 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:06.094671 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 05:01:06.094840 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 05:01:06.095793 28008 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:06.190156 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 27803
I20260809 05:01:06.217967 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 27247
I20260809 05:01:06.219310 27333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:06.333559 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 27247
2026-08-09T05:01:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4754 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T05:01:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:06Z Disabled control of system clock
I20260809 05:01:06.430382 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41359
--webserver_interface=127.29.243.254
--webserver_port=0
--builtin_ntp_servers=127.29.243.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:01:06.769526 28095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:06.769804 28095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:06.769923 28095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:06.777033 28095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:06.777132 28095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:06.777189 28095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:06.777268 28095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:06.777326 28095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:06.786567 28095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41359
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:06.787909 28095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:06.789813 28095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:06.801342 28105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:06.801403 28106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:06.801383 28109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:06.802364 28095 server_base.cc:1034] running on GCE node
I20260809 05:01:06.803136 28095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:06.804524 28095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:06.805732 28095 hybrid_clock.cc:648] HybridClock initialized: now 1786251666805679 us; error 49 us; skew 500 ppm
I20260809 05:01:06.806074 28095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:01:06.808331 28095 webserver.cc:460] Webserver started at http://127.29.243.254:33147/ using document root <none> and password file <none>
I20260809 05:01:06.809013 28095 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:06.809134 28095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:06.809444 28095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:06.812386 28095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "fbb5b141e9eb494d891ac568b5fb8a3c"
format_stamp: "Formatted at 2026-08-09 05:01:06 on dist-test-slave-21b9"
server_key: "825337d35467051959208166c46a428e"
server_key_iv: "926cf40520b986ba690e700d09da47bb"
server_key_version: "encryptionkey@0"
I20260809 05:01:06.813179 28095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "fbb5b141e9eb494d891ac568b5fb8a3c"
format_stamp: "Formatted at 2026-08-09 05:01:06 on dist-test-slave-21b9"
server_key: "825337d35467051959208166c46a428e"
server_key_iv: "926cf40520b986ba690e700d09da47bb"
server_key_version: "encryptionkey@0"
I20260809 05:01:06.818315 28095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:01:06.821797 28115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:06.823175 28095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:01:06.823380 28095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "fbb5b141e9eb494d891ac568b5fb8a3c"
format_stamp: "Formatted at 2026-08-09 05:01:06 on dist-test-slave-21b9"
server_key: "825337d35467051959208166c46a428e"
server_key_iv: "926cf40520b986ba690e700d09da47bb"
server_key_version: "encryptionkey@0"
I20260809 05:01:06.823591 28095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:06.841627 28095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:06.842427 28095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:06.842687 28095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:06.860114 28095 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:41359
I20260809 05:01:06.860134 28194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:41359 every 8 connection(s)
I20260809 05:01:06.861426 28095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:06.865195 28197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:06.866851 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 28095
I20260809 05:01:06.867017 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:06.867486 30671 external_mini_cluster.cc:1428] Setting key a8791df97e4d2f33730aab4cee4068a4
I20260809 05:01:06.876883 28197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c: Bootstrap starting.
I20260809 05:01:06.880431 28197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:06.881462 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:06.869406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51588 (local address 127.29.243.254:41359)
0809 05:01:06.870414 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:06.870425 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:06.870469 (+ 44us) server_negotiation.cc:407] Connection header received
0809 05:01:06.872221 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:06.872245 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:06.872680 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:06.872978 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:06.873664 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:06.875085 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:06.876981 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:06.877699 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:06.877828 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:06.879080 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:06.879089 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:06.879102 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:06.879724 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:06.879813 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:06.879835 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:06.879918 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:06.880148 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 05:01:06.880507 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":326,"threads_started":1}
I20260809 05:01:06.881443 28197 log.cc:824] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:06.884330 28197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c: No bootstrap required, opened a new log
I20260809 05:01:06.891516 28197 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb5b141e9eb494d891ac568b5fb8a3c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41359 } }
I20260809 05:01:06.891836 28197 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:06.891937 28197 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb5b141e9eb494d891ac568b5fb8a3c, State: Initialized, Role: FOLLOWER
I20260809 05:01:06.892539 28197 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb5b141e9eb494d891ac568b5fb8a3c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41359 } }
I20260809 05:01:06.892773 28197 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:06.892926 28197 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:06.893085 28197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:06.895078 28197 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb5b141e9eb494d891ac568b5fb8a3c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41359 } }
I20260809 05:01:06.895550 28197 leader_election.cc:302] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbb5b141e9eb494d891ac568b5fb8a3c; no voters:
I20260809 05:01:06.896086 28197 leader_election.cc:288] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:06.896234 28205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:06.898025 28197 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:06.898370 28205 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [term 1 LEADER]: Becoming Leader. State: Replica: fbb5b141e9eb494d891ac568b5fb8a3c, State: Running, Role: LEADER
I20260809 05:01:06.898842 28205 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb5b141e9eb494d891ac568b5fb8a3c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41359 } }
I20260809 05:01:06.901916 28206 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbb5b141e9eb494d891ac568b5fb8a3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb5b141e9eb494d891ac568b5fb8a3c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41359 } } }
I20260809 05:01:06.902402 28206 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:06.901949 28207 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbb5b141e9eb494d891ac568b5fb8a3c. Latest consensus state: current_term: 1 leader_uuid: "fbb5b141e9eb494d891ac568b5fb8a3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb5b141e9eb494d891ac568b5fb8a3c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41359 } } }
I20260809 05:01:06.902765 28207 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c [sys.catalog]: This master's current role is: LEADER
W20260809 05:01:06.908805 28228 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:01:06.908933 28228 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:01:06.909099 28222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:06.910784 28222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:06.918609 28222 catalog_manager.cc:1347] Generated new cluster ID: ae8edc032f5b41a9b7ba061a7d642a64
I20260809 05:01:06.918709 28222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:06.936355 28222 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:06.937626 28222 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:06.948136 28222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c: Generated new TSK 0
I20260809 05:01:06.948915 28222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:06.955083 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41359
--builtin_ntp_servers=127.29.243.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:01:07.311470 28231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:07.311753 28231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:07.311920 28231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:07.319499 28231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:07.319615 28231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:07.319778 28231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 05:01:07.329320 28231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:07.330724 28231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:07.332795 28231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:07.346411 28241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:07.346480 28240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:07.346513 28246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:07.347080 28231 server_base.cc:1034] running on GCE node
I20260809 05:01:07.347752 28231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:07.348744 28231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:07.349953 28231 hybrid_clock.cc:648] HybridClock initialized: now 1786251667349908 us; error 32 us; skew 500 ppm
I20260809 05:01:07.350272 28231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:01:07.352622 28231 webserver.cc:460] Webserver started at http://127.29.243.193:42133/ using document root <none> and password file <none>
I20260809 05:01:07.353331 28231 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:07.353447 28231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:07.353765 28231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:07.356726 28231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "1adc54c85c2e4ac9afcd94059b812312"
format_stamp: "Formatted at 2026-08-09 05:01:07 on dist-test-slave-21b9"
server_key: "1d708bccece5029d2ccce6e9b461a385"
server_key_iv: "abaf26113ceb1ef6fc6c40ef90d1ad45"
server_key_version: "encryptionkey@0"
I20260809 05:01:07.357506 28231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "1adc54c85c2e4ac9afcd94059b812312"
format_stamp: "Formatted at 2026-08-09 05:01:07 on dist-test-slave-21b9"
server_key: "1d708bccece5029d2ccce6e9b461a385"
server_key_iv: "abaf26113ceb1ef6fc6c40ef90d1ad45"
server_key_version: "encryptionkey@0"
I20260809 05:01:07.362709 28231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:01:07.366268 28252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:07.367694 28231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:07.367870 28231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "1adc54c85c2e4ac9afcd94059b812312"
format_stamp: "Formatted at 2026-08-09 05:01:07 on dist-test-slave-21b9"
server_key: "1d708bccece5029d2ccce6e9b461a385"
server_key_iv: "abaf26113ceb1ef6fc6c40ef90d1ad45"
server_key_version: "encryptionkey@0"
I20260809 05:01:07.368094 28231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:07.387455 28231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:07.388324 28231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:07.388762 28231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:07.389880 28231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:07.391471 28231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:07.391571 28231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:07.391676 28231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:07.391753 28231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:07.422222 28231 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:43425
I20260809 05:01:07.422242 28429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:43425 every 8 connection(s)
I20260809 05:01:07.423580 28231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 05:01:07.427398 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 28231
I20260809 05:01:07.427598 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 05:01:07.428068 30671 external_mini_cluster.cc:1428] Setting key 375aa1e6c6cf28b706e6ccc39e4b89af
I20260809 05:01:07.430722 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41359
--builtin_ntp_servers=127.29.243.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:01:07.436419 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:07.426227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:35459 (local address 127.29.243.254:41359)
0809 05:01:07.426676 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:07.426681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:07.430015 (+ 3334us) server_negotiation.cc:407] Connection header received
0809 05:01:07.430070 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:07.430074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:07.430138 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:07.430228 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:07.431391 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:07.432134 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:07.433668 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:07.434120 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:07.434234 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:07.435577 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:07.435582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:07.435586 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:07.435603 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:07.435645 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:07.435657 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:07.435731 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:07.435957 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 05:01:07.436067 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":227,"threads_started":1}
I20260809 05:01:07.437371 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:07.426819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41359 (local address 127.29.243.193:35459)
0809 05:01:07.427963 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:07.428033 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:07.429724 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:07.430487 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:07.430505 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:07.430594 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:07.431207 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:07.431225 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:07.432283 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:07.432288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:07.433528 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:07.433542 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:07.434344 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:07.434351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:07.434594 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:07.435237 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:01:07.435240 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:07.435305 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:01:07.435394 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:07.435743 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:07.435755 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:07.435764 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 05:01:07.435914 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 05:01:07.436298 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":372,"threads_started":1}
I20260809 05:01:07.438800 28431 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41359
I20260809 05:01:07.439105 28431 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:07.439888 28431 heartbeater.cc:511] Master 127.29.243.254:41359 requested a full tablet report, sending...
I20260809 05:01:07.441926 28149 ts_manager.cc:194] Registered new tserver with Master: 1adc54c85c2e4ac9afcd94059b812312 (127.29.243.193:43425)
I20260809 05:01:07.443470 28149 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.193:35459
W20260809 05:01:07.777580 28436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:07.777884 28436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:07.778008 28436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:07.785156 28436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:07.785261 28436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:07.785404 28436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 05:01:07.794457 28436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:07.795871 28436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:07.797775 28436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:07.811044 28444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:07.811110 28441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:07.811136 28442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:07.812557 28436 server_base.cc:1034] running on GCE node
I20260809 05:01:07.813102 28436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:07.814023 28436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:07.815240 28436 hybrid_clock.cc:648] HybridClock initialized: now 1786251667815174 us; error 53 us; skew 500 ppm
I20260809 05:01:07.815587 28436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:01:07.817799 28436 webserver.cc:460] Webserver started at http://127.29.243.194:32823/ using document root <none> and password file <none>
I20260809 05:01:07.818506 28436 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:07.818616 28436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:07.818938 28436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:07.821835 28436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "50f2f8033e9647738f1fa28879f2ce74"
format_stamp: "Formatted at 2026-08-09 05:01:07 on dist-test-slave-21b9"
server_key: "bc181024ce1a23ccce75409d95e22bdd"
server_key_iv: "2b2fb77c04bd788af449b0ddaad089e5"
server_key_version: "encryptionkey@0"
I20260809 05:01:07.822609 28436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "50f2f8033e9647738f1fa28879f2ce74"
format_stamp: "Formatted at 2026-08-09 05:01:07 on dist-test-slave-21b9"
server_key: "bc181024ce1a23ccce75409d95e22bdd"
server_key_iv: "2b2fb77c04bd788af449b0ddaad089e5"
server_key_version: "encryptionkey@0"
I20260809 05:01:07.827723 28436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:01:07.831216 28454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:07.832579 28436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:01:07.832734 28436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "50f2f8033e9647738f1fa28879f2ce74"
format_stamp: "Formatted at 2026-08-09 05:01:07 on dist-test-slave-21b9"
server_key: "bc181024ce1a23ccce75409d95e22bdd"
server_key_iv: "2b2fb77c04bd788af449b0ddaad089e5"
server_key_version: "encryptionkey@0"
I20260809 05:01:07.832939 28436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:07.870467 28436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:07.871367 28436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:07.871660 28436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:07.872738 28436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:07.874277 28436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:07.874377 28436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:07.874497 28436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:07.874568 28436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:07.905355 28436 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:38475
I20260809 05:01:07.905405 28626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:38475 every 8 connection(s)
I20260809 05:01:07.906666 28436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 05:01:07.912719 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 28436
I20260809 05:01:07.912925 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 05:01:07.913466 30671 external_mini_cluster.cc:1428] Setting key 96323a0ee43009e6e45f6ab7bfc801f7
I20260809 05:01:07.916478 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41359
--builtin_ntp_servers=127.29.243.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:01:07.919750 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:07.909289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:44937 (local address 127.29.243.254:41359)
0809 05:01:07.909503 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:07.909509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:07.910898 (+ 1389us) server_negotiation.cc:407] Connection header received
0809 05:01:07.912929 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:07.912935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:07.913022 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:07.913108 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:07.914539 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:07.915547 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:07.917150 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:07.917548 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:07.917657 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:07.918944 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:07.918949 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:07.918952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:07.918970 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:07.919002 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:07.919013 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:07.919085 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:07.919321 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 05:01:07.919441 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 05:01:07.920749 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:07.909799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41359 (local address 127.29.243.194:44937)
0809 05:01:07.910739 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:07.910775 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:07.912579 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:07.913430 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:07.913442 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:07.913567 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:07.914346 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:07.914372 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:07.915701 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:07.915705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:07.917008 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:07.917022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:07.917770 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:07.917779 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:07.918002 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:07.918647 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:01:07.918649 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:07.918688 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:01:07.918766 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:07.919099 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:07.919111 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:07.919120 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 05:01:07.919294 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 05:01:07.919685 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":273,"threads_started":1}
I20260809 05:01:07.922044 28627 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41359
I20260809 05:01:07.922360 28627 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:07.923292 28627 heartbeater.cc:511] Master 127.29.243.254:41359 requested a full tablet report, sending...
I20260809 05:01:07.924799 28149 ts_manager.cc:194] Registered new tserver with Master: 50f2f8033e9647738f1fa28879f2ce74 (127.29.243.194:38475)
I20260809 05:01:07.925650 28149 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.194:44937
W20260809 05:01:08.260164 28636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:08.260444 28636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:08.260558 28636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:08.267767 28636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:08.267874 28636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:08.268014 28636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 05:01:08.277372 28636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:41359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:08.278733 28636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:08.280618 28636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:08.293613 28646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:08.293700 28642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:08.293686 28643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:08.294799 28636 server_base.cc:1034] running on GCE node
I20260809 05:01:08.295429 28636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:08.296373 28636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:08.297609 28636 hybrid_clock.cc:648] HybridClock initialized: now 1786251668297567 us; error 33 us; skew 500 ppm
I20260809 05:01:08.297945 28636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:01:08.300369 28636 webserver.cc:460] Webserver started at http://127.29.243.195:42475/ using document root <none> and password file <none>
I20260809 05:01:08.301091 28636 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:08.301180 28636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:08.301493 28636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:08.304419 28636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "a0cbe15dfbe04f4fba134a13e48ed55f"
format_stamp: "Formatted at 2026-08-09 05:01:08 on dist-test-slave-21b9"
server_key: "93e37ba2e0d57eea5143f2ce742237c9"
server_key_iv: "a5d401c5d1fcdbb692eef49c4337a8ba"
server_key_version: "encryptionkey@0"
I20260809 05:01:08.305163 28636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "a0cbe15dfbe04f4fba134a13e48ed55f"
format_stamp: "Formatted at 2026-08-09 05:01:08 on dist-test-slave-21b9"
server_key: "93e37ba2e0d57eea5143f2ce742237c9"
server_key_iv: "a5d401c5d1fcdbb692eef49c4337a8ba"
server_key_version: "encryptionkey@0"
I20260809 05:01:08.310138 28636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260809 05:01:08.313678 28656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:08.315047 28636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:01:08.315178 28636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "a0cbe15dfbe04f4fba134a13e48ed55f"
format_stamp: "Formatted at 2026-08-09 05:01:08 on dist-test-slave-21b9"
server_key: "93e37ba2e0d57eea5143f2ce742237c9"
server_key_iv: "a5d401c5d1fcdbb692eef49c4337a8ba"
server_key_version: "encryptionkey@0"
I20260809 05:01:08.315399 28636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:08.328410 28636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:08.329190 28636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:08.329442 28636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:08.330327 28636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:08.331831 28636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:08.331918 28636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:08.332010 28636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:08.332065 28636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:08.362813 28636 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:43533
I20260809 05:01:08.362859 28851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:43533 every 8 connection(s)
I20260809 05:01:08.364388 28636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 05:01:08.367540 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 28636
I20260809 05:01:08.367729 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 05:01:08.368321 30671 external_mini_cluster.cc:1428] Setting key b9c95188caff54c07b69d8e45e081de3
I20260809 05:01:08.376988 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.366959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:55483 (local address 127.29.243.254:41359)
0809 05:01:08.367161 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:08.367166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:08.368985 (+ 1819us) server_negotiation.cc:407] Connection header received
0809 05:01:08.370766 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:08.370771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:08.370855 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:08.370964 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:08.372368 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.373154 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.374632 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.375033 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.375122 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.376361 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:08.376366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:08.376370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:08.376387 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:08.376420 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:08.376432 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:08.376501 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:08.376672 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 05:01:08.376747 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 05:01:08.377878 28857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.367654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41359 (local address 127.29.243.195:55483)
0809 05:01:08.368693 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.368733 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:08.370521 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:08.371451 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:08.371473 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:08.371581 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:08.372217 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.372237 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.373276 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.373280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.374516 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.374529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.375200 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.375204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.375479 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.376079 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:01:08.376081 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:08.376119 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:01:08.376192 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:08.376512 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:08.376520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:08.376525 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 05:01:08.376649 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 05:01:08.376914 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":321,"threads_started":1}
I20260809 05:01:08.379050 28855 heartbeater.cc:347] Connected to a master server at 127.29.243.254:41359
I20260809 05:01:08.379348 28855 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:08.380088 28855 heartbeater.cc:511] Master 127.29.243.254:41359 requested a full tablet report, sending...
I20260809 05:01:08.381474 28149 ts_manager.cc:194] Registered new tserver with Master: a0cbe15dfbe04f4fba134a13e48ed55f (127.29.243.195:43533)
I20260809 05:01:08.382378 28149 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.195:55483
I20260809 05:01:08.383428 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:01:08.392702 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.387238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51602 (local address 127.29.243.254:41359)
0809 05:01:08.387469 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:08.387476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:08.387821 (+ 345us) server_negotiation.cc:407] Connection header received
0809 05:01:08.387977 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:08.387981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:08.388038 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:08.388116 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:08.388537 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.389218 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.390478 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.390882 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.390983 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.392043 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:08.392048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:08.392051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:08.392067 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:08.392097 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:08.392109 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:08.392221 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:08.392340 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 05:01:08.392391 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 05:01:08.396476 28147 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:01:08.403357 28147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:01:08.427098 28874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.415677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.29.243.193:43425)
0809 05:01:08.416690 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:08.416698 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:08.417367 (+ 669us) server_negotiation.cc:407] Connection header received
0809 05:01:08.417637 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:08.417646 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:08.417890 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:08.418072 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:08.419151 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.420740 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.424054 (+ 3314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.425480 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.425650 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.425854 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:08.426519 (+ 665us) server_negotiation.cc:299] Negotiation successful
0809 05:01:08.426703 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":731,"threads_started":1}
I20260809 05:01:08.428143 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.415383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.29.243.195:43533)
0809 05:01:08.416246 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:08.416254 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:08.417302 (+ 1048us) server_negotiation.cc:407] Connection header received
0809 05:01:08.417638 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:08.417646 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:08.417884 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:08.418052 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:08.419169 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.420773 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.424181 (+ 3408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.425626 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.425791 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.426003 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:08.427577 (+ 1574us) server_negotiation.cc:299] Negotiation successful
0809 05:01:08.427746 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":514,"threads_started":1}
I20260809 05:01:08.428170 28879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.415766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43425 (local address 127.0.0.1:60570)
0809 05:01:08.417103 (+ 1337us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.417180 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:08.417371 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:08.418347 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:08.418355 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:08.418387 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:08.418903 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.418922 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.420908 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.420914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.423778 (+ 2864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.423794 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.426044 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.426050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.426280 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.426327 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 05:01:08.427585 (+ 1258us) client_negotiation.cc:240] Negotiation successful
0809 05:01:08.427776 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1137,"mutex_wait_us":466,"spinlock_wait_cycles":9600,"thread_start_us":1345,"threads_started":1}
I20260809 05:01:08.429190 28877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.415243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:43533 (local address 127.0.0.1:36406)
0809 05:01:08.417103 (+ 1860us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.417180 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:08.417371 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:08.418378 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:08.418382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:08.418400 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:08.418979 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.418989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.421004 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.421008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.423950 (+ 2942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.423962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.426196 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.426203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.427365 (+ 1162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.427403 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 05:01:08.428623 (+ 1220us) client_negotiation.cc:240] Negotiation successful
0809 05:01:08.428759 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1556,"mutex_wait_us":29,"spinlock_wait_cycles":6272,"thread_start_us":1374,"threads_started":1}
I20260809 05:01:08.432507 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.417482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:38475 (local address 127.0.0.1:48512)
0809 05:01:08.421866 (+ 4384us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.421880 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:08.422012 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:08.423908 (+ 1896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:08.423912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:08.423938 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:08.424325 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.424337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.428034 (+ 3697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.428046 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.429980 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.429993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.431804 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.431811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.431976 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.432022 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 05:01:08.432114 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 05:01:08.432234 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4265,"thread_start_us":4200,"threads_started":1}
I20260809 05:01:08.431995 28328 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4a3b34cf904e239fea94cd84ff10ba (DEFAULT_TABLE table=test-table [id=9b905b701d1f41ba98f3cdca8de53bc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:08.433468 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.417887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48512 (local address 127.29.243.194:38475)
0809 05:01:08.421485 (+ 3598us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:08.421492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:08.421982 (+ 490us) server_negotiation.cc:407] Connection header received
0809 05:01:08.422181 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:08.422189 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:08.423532 (+ 1343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:08.423706 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:08.424535 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.426161 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.430162 (+ 4001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.431583 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.431742 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.431957 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:08.432815 (+ 858us) server_negotiation.cc:299] Negotiation successful
0809 05:01:08.432984 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3426,"thread_start_us":3300,"threads_started":1}
I20260809 05:01:08.433643 28328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4a3b34cf904e239fea94cd84ff10ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:08.433171 28744 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4a3b34cf904e239fea94cd84ff10ba (DEFAULT_TABLE table=test-table [id=9b905b701d1f41ba98f3cdca8de53bc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:08.434403 28744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4a3b34cf904e239fea94cd84ff10ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:08.437198 28532 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4a3b34cf904e239fea94cd84ff10ba (DEFAULT_TABLE table=test-table [id=9b905b701d1f41ba98f3cdca8de53bc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:08.438525 28532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4a3b34cf904e239fea94cd84ff10ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:08.446411 28431 heartbeater.cc:503] Master 127.29.243.254:41359 was elected leader, sending a full tablet report...
I20260809 05:01:08.448021 28883 tablet_bootstrap.cc:492] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312: Bootstrap starting.
I20260809 05:01:08.451663 28883 tablet_bootstrap.cc:654] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:08.452697 28883 log.cc:824] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:08.453308 28885 tablet_bootstrap.cc:492] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74: Bootstrap starting.
I20260809 05:01:08.454768 28884 tablet_bootstrap.cc:492] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f: Bootstrap starting.
I20260809 05:01:08.457180 28883 tablet_bootstrap.cc:492] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312: No bootstrap required, opened a new log
I20260809 05:01:08.457469 28883 ts_tablet_manager.cc:1397] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260809 05:01:08.457820 28885 tablet_bootstrap.cc:654] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:08.458354 28884 tablet_bootstrap.cc:654] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:08.458801 28885 log.cc:824] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:08.459574 28884 log.cc:824] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:08.461594 28885 tablet_bootstrap.cc:492] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74: No bootstrap required, opened a new log
I20260809 05:01:08.461839 28885 ts_tablet_manager.cc:1397] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260809 05:01:08.462584 28884 tablet_bootstrap.cc:492] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f: No bootstrap required, opened a new log
I20260809 05:01:08.462848 28884 ts_tablet_manager.cc:1397] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260809 05:01:08.467460 28883 raft_consensus.cc:348] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43533 } } peers { permanent_uuid: "1adc54c85c2e4ac9afcd94059b812312" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43425 } } peers { permanent_uuid: "50f2f8033e9647738f1fa28879f2ce74" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38475 } }
I20260809 05:01:08.467945 28883 raft_consensus.cc:374] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:08.468075 28883 raft_consensus.cc:729] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1adc54c85c2e4ac9afcd94059b812312, State: Initialized, Role: FOLLOWER
I20260809 05:01:08.468756 28883 consensus_queue.cc:260] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43533 } } peers { permanent_uuid: "1adc54c85c2e4ac9afcd94059b812312" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43425 } } peers { permanent_uuid: "50f2f8033e9647738f1fa28879f2ce74" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38475 } }
I20260809 05:01:08.469594 28885 raft_consensus.cc:348] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43533 } } peers { permanent_uuid: "1adc54c85c2e4ac9afcd94059b812312" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43425 } } peers { permanent_uuid: "50f2f8033e9647738f1fa28879f2ce74" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38475 } }
I20260809 05:01:08.470618 28883 ts_tablet_manager.cc:1428] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260809 05:01:08.470643 28885 raft_consensus.cc:374] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:08.470829 28885 raft_consensus.cc:729] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f2f8033e9647738f1fa28879f2ce74, State: Initialized, Role: FOLLOWER
I20260809 05:01:08.471592 28885 consensus_queue.cc:260] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43533 } } peers { permanent_uuid: "1adc54c85c2e4ac9afcd94059b812312" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43425 } } peers { permanent_uuid: "50f2f8033e9647738f1fa28879f2ce74" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38475 } }
I20260809 05:01:08.472836 28627 heartbeater.cc:503] Master 127.29.243.254:41359 was elected leader, sending a full tablet report...
I20260809 05:01:08.472689 28884 raft_consensus.cc:348] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43533 } } peers { permanent_uuid: "1adc54c85c2e4ac9afcd94059b812312" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43425 } } peers { permanent_uuid: "50f2f8033e9647738f1fa28879f2ce74" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38475 } }
I20260809 05:01:08.473129 28884 raft_consensus.cc:374] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:08.473232 28884 raft_consensus.cc:729] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0cbe15dfbe04f4fba134a13e48ed55f, State: Initialized, Role: FOLLOWER
I20260809 05:01:08.473551 28885 ts_tablet_manager.cc:1428] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260809 05:01:08.473788 28884 consensus_queue.cc:260] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43533 } } peers { permanent_uuid: "1adc54c85c2e4ac9afcd94059b812312" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43425 } } peers { permanent_uuid: "50f2f8033e9647738f1fa28879f2ce74" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38475 } }
I20260809 05:01:08.474802 28855 heartbeater.cc:503] Master 127.29.243.254:41359 was elected leader, sending a full tablet report...
I20260809 05:01:08.476056 28884 ts_tablet_manager.cc:1428] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f: Time spent starting tablet: real 0.013s user 0.012s sys 0.002s
I20260809 05:01:08.512142 28895 raft_consensus.cc:482] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:01:08.512421 28895 raft_consensus.cc:504] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43533 } } peers { permanent_uuid: "1adc54c85c2e4ac9afcd94059b812312" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43425 } } peers { permanent_uuid: "50f2f8033e9647738f1fa28879f2ce74" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38475 } }
I20260809 05:01:08.514145 28895 leader_election.cc:288] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1adc54c85c2e4ac9afcd94059b812312 (127.29.243.193:43425), 50f2f8033e9647738f1fa28879f2ce74 (127.29.243.194:38475)
I20260809 05:01:08.520709 28857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.514552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43425 (local address 127.29.243.195:34673)
0809 05:01:08.514742 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.514762 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:08.514902 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:08.515582 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:08.515586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:08.515616 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:08.515816 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.515830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.516996 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.516999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.518850 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.518876 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.520166 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.520171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.520287 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.520312 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 05:01:08.520393 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 05:01:08.520488 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 05:01:08.520802 28874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.514581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:34673 (local address 127.29.243.193:43425)
0809 05:01:08.514802 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:08.514807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:08.514831 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:01:08.515131 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:08.515138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:08.515229 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:08.515346 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:08.516017 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.516886 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.519010 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.520029 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.520175 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.520246 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:08.520427 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 05:01:08.520562 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 05:01:08.521514 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.514983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:38475 (local address 127.29.243.195:52691)
0809 05:01:08.515691 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.515705 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:08.515824 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:08.516194 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:08.516199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:08.516225 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:08.516496 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.516503 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.517576 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.517582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.519424 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:08.519436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.520860 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.520866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:08.521044 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.521076 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 05:01:08.521186 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 05:01:08.521270 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":398,"threads_started":1}
I20260809 05:01:08.521570 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.515150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:52691 (local address 127.29.243.194:38475)
0809 05:01:08.515433 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:08.515439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:08.515787 (+ 348us) server_negotiation.cc:407] Connection header received
0809 05:01:08.515997 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:08.516003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:08.516083 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:08.516166 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:08.516642 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.517428 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.519594 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.520684 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.520844 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.520907 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:08.521229 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 05:01:08.521354 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260809 05:01:08.521979 28362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4a3b34cf904e239fea94cd84ff10ba" candidate_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1adc54c85c2e4ac9afcd94059b812312" is_pre_election: true
I20260809 05:01:08.522538 28362 raft_consensus.cc:2454] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0cbe15dfbe04f4fba134a13e48ed55f in term 0.
I20260809 05:01:08.522526 28556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4a3b34cf904e239fea94cd84ff10ba" candidate_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50f2f8033e9647738f1fa28879f2ce74" is_pre_election: true
I20260809 05:01:08.522989 28556 raft_consensus.cc:2454] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0cbe15dfbe04f4fba134a13e48ed55f in term 0.
I20260809 05:01:08.523145 28657 leader_election.cc:302] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1adc54c85c2e4ac9afcd94059b812312, a0cbe15dfbe04f4fba134a13e48ed55f; no voters:
I20260809 05:01:08.523550 28895 raft_consensus.cc:2781] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:01:08.523696 28895 raft_consensus.cc:482] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:01:08.523792 28895 raft_consensus.cc:3037] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:08.525928 28895 raft_consensus.cc:504] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43533 } } peers { permanent_uuid: "1adc54c85c2e4ac9afcd94059b812312" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43425 } } peers { permanent_uuid: "50f2f8033e9647738f1fa28879f2ce74" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38475 } }
I20260809 05:01:08.526476 28895 leader_election.cc:288] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [CANDIDATE]: Term 1 election: Requested vote from peers 1adc54c85c2e4ac9afcd94059b812312 (127.29.243.193:43425), 50f2f8033e9647738f1fa28879f2ce74 (127.29.243.194:38475)
I20260809 05:01:08.526968 28362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4a3b34cf904e239fea94cd84ff10ba" candidate_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1adc54c85c2e4ac9afcd94059b812312"
I20260809 05:01:08.527038 28556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4a3b34cf904e239fea94cd84ff10ba" candidate_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50f2f8033e9647738f1fa28879f2ce74"
I20260809 05:01:08.527153 28362 raft_consensus.cc:3037] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:08.527235 28556 raft_consensus.cc:3037] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:08.529415 28362 raft_consensus.cc:2454] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0cbe15dfbe04f4fba134a13e48ed55f in term 1.
I20260809 05:01:08.529624 28556 raft_consensus.cc:2454] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0cbe15dfbe04f4fba134a13e48ed55f in term 1.
I20260809 05:01:08.529789 28657 leader_election.cc:302] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1adc54c85c2e4ac9afcd94059b812312, a0cbe15dfbe04f4fba134a13e48ed55f; no voters:
I20260809 05:01:08.530064 28895 raft_consensus.cc:2781] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:08.530522 28895 raft_consensus.cc:686] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 1 LEADER]: Becoming Leader. State: Replica: a0cbe15dfbe04f4fba134a13e48ed55f, State: Running, Role: LEADER
I20260809 05:01:08.530987 28895 consensus_queue.cc:237] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43533 } } peers { permanent_uuid: "1adc54c85c2e4ac9afcd94059b812312" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43425 } } peers { permanent_uuid: "50f2f8033e9647738f1fa28879f2ce74" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38475 } }
I20260809 05:01:08.535831 28147 catalog_manager.cc:5495] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f reported cstate change: term changed from 0 to 1, leader changed from <none> to a0cbe15dfbe04f4fba134a13e48ed55f (127.29.243.195). New cstate: current_term: 1 leader_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cbe15dfbe04f4fba134a13e48ed55f" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 43533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1adc54c85c2e4ac9afcd94059b812312" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50f2f8033e9647738f1fa28879f2ce74" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38475 } health_report { overall_health: UNKNOWN } } }
I20260809 05:01:08.551524 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.546085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36422 (local address 127.29.243.195:43533)
0809 05:01:08.546307 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:08.546313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:08.546449 (+ 136us) server_negotiation.cc:407] Connection header received
0809 05:01:08.546647 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:08.546652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:08.546726 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:08.546868 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:01:08.547247 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.548015 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.549383 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.549782 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.549899 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.550464 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:01:08.550942 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:01:08.551106 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:08.551239 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 05:01:08.551311 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 05:01:08.559329 28556 raft_consensus.cc:1261] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74 [term 1 FOLLOWER]: Refusing update from remote peer a0cbe15dfbe04f4fba134a13e48ed55f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:08.559391 28362 raft_consensus.cc:1261] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312 [term 1 FOLLOWER]: Refusing update from remote peer a0cbe15dfbe04f4fba134a13e48ed55f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:08.560266 28899 consensus_queue.cc:1035] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50f2f8033e9647738f1fa28879f2ce74" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 38475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:01:08.560645 28895 consensus_queue.cc:1035] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1adc54c85c2e4ac9afcd94059b812312" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 43425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:01:08.590436 28149 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:01:08.590663 28149 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:01:08.594437 28149 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbb5b141e9eb494d891ac568b5fb8a3c: Sending DeleteTablet for 3 replicas of tablet 0e4a3b34cf904e239fea94cd84ff10ba
I20260809 05:01:08.595501 28744 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e4a3b34cf904e239fea94cd84ff10ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:08 UTC) from {username='slave'} at 127.0.0.1:36406
I20260809 05:01:08.595738 28328 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e4a3b34cf904e239fea94cd84ff10ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:08 UTC) from {username='slave'} at 127.0.0.1:60570
I20260809 05:01:08.596212 28532 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e4a3b34cf904e239fea94cd84ff10ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:08 UTC) from {username='slave'} at 127.0.0.1:48512
I20260809 05:01:08.596742 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 28231
I20260809 05:01:08.610245 28923 tablet_replica.cc:330] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74: stopping tablet replica
I20260809 05:01:08.611349 28923 raft_consensus.cc:2229] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:01:08.612196 28923 raft_consensus.cc:2258] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:08.614660 28923 ts_tablet_manager.cc:1905] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:08.618468 28922 tablet_replica.cc:330] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312: stopping tablet replica
I20260809 05:01:08.618765 28921 tablet_replica.cc:330] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f: stopping tablet replica
I20260809 05:01:08.618914 28922 raft_consensus.cc:2229] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:01:08.619544 28922 raft_consensus.cc:2258] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:08.620433 28921 raft_consensus.cc:2229] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:01:08.621021 28921 raft_consensus.cc:2258] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:08.622711 28923 ts_tablet_manager.cc:1918] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:08.622922 28923 log.cc:1196] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/0e4a3b34cf904e239fea94cd84ff10ba
I20260809 05:01:08.623373 28921 ts_tablet_manager.cc:1905] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:08.623553 28922 ts_tablet_manager.cc:1905] T 0e4a3b34cf904e239fea94cd84ff10ba P 1adc54c85c2e4ac9afcd94059b812312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:08.623579 28923 ts_tablet_manager.cc:1939] T 0e4a3b34cf904e239fea94cd84ff10ba P 50f2f8033e9647738f1fa28879f2ce74: Deleting consensus metadata
I20260809 05:01:08.625456 28116 catalog_manager.cc:4841] TS 50f2f8033e9647738f1fa28879f2ce74 (127.29.243.194:38475): tablet 0e4a3b34cf904e239fea94cd84ff10ba (table test-table [id=9b905b701d1f41ba98f3cdca8de53bc7]) successfully deleted
I20260809 05:01:08.625706 28874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.597593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.29.243.193:43425)
0809 05:01:08.597794 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:08.597801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:08.611767 (+ 13966us) server_negotiation.cc:407] Connection header received
0809 05:01:08.612643 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:08.612649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:08.612766 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:08.612850 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:08.614049 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.615200 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.619068 (+ 3868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.619773 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.619898 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.624107 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:08.624119 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:08.624136 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:08.624832 (+ 696us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:08.624914 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:08.624940 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:08.625023 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:08.625265 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 05:01:08.625380 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 05:01:08.627172 28422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:08.631464 28921 ts_tablet_manager.cc:1918] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:08.631621 28921 log.cc:1196] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/0e4a3b34cf904e239fea94cd84ff10ba
I20260809 05:01:08.632074 28921 ts_tablet_manager.cc:1939] T 0e4a3b34cf904e239fea94cd84ff10ba P a0cbe15dfbe04f4fba134a13e48ed55f: Deleting consensus metadata
I20260809 05:01:08.633468 28119 catalog_manager.cc:4841] TS a0cbe15dfbe04f4fba134a13e48ed55f (127.29.243.195:43533): tablet 0e4a3b34cf904e239fea94cd84ff10ba (table test-table [id=9b905b701d1f41ba98f3cdca8de53bc7]) successfully deleted
I20260809 05:01:08.736616 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 28231
I20260809 05:01:08.766191 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 28436
W20260809 05:01:08.766230 28116 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.243.193:43425 (error 108)
I20260809 05:01:08.768487 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.767894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43425 (local address 127.0.0.1:60590)
0809 05:01:08.768162 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.768280 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260809 05:01:08.768774 28116 catalog_manager.cc:4584] TS 1adc54c85c2e4ac9afcd94059b812312 (127.29.243.193:43425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e4a3b34cf904e239fea94cd84ff10ba: Network error: Client connection negotiation failed: client connection to 127.29.243.193:43425: connect: Connection refused (error 111)
I20260809 05:01:08.773557 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.767186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.29.243.194:38475)
0809 05:01:08.767418 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:08.767424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:08.767536 (+ 112us) server_negotiation.cc:407] Connection header received
0809 05:01:08.767738 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:08.767745 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:08.767853 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:08.767945 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:08.768403 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.769290 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.770555 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.770956 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.771072 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.772184 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:08.772194 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:08.772210 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:08.772813 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:08.772869 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:08.772890 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:08.772993 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:08.773141 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 05:01:08.773254 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 05:01:08.774253 28619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:08.822486 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.821896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43425 (local address 127.0.0.1:60602)
0809 05:01:08.822160 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.822286 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 05:01:08.823668 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.823224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43425 (local address 127.0.0.1:60612)
0809 05:01:08.823407 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.823500 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 05:01:08.862599 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 28436
I20260809 05:01:08.884616 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.884003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43425 (local address 127.0.0.1:60614)
0809 05:01:08.884284 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.884386 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260809 05:01:08.885812 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.885358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43425 (local address 127.0.0.1:60626)
0809 05:01:08.885582 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.885661 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260809 05:01:08.890792 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 28636
I20260809 05:01:08.898582 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.891542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36436 (local address 127.29.243.195:43533)
0809 05:01:08.891717 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:08.891722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:08.891740 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:01:08.891947 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:08.891951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:08.892043 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:08.892129 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:08.892599 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.893505 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.895443 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:08.895891 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:08.896004 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:08.897145 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:08.897155 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:08.897171 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:08.897792 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:08.897841 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:08.897861 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:08.897946 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:08.898111 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 05:01:08.898191 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 05:01:08.899385 28847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:08.966244 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.965632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43425 (local address 127.0.0.1:60630)
0809 05:01:08.965910 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.966044 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 05:01:08.967494 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:08.967000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43425 (local address 127.0.0.1:60638)
0809 05:01:08.967228 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:08.967351 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260809 05:01:09.020908 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 28636
I20260809 05:01:09.050741 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 28095
I20260809 05:01:09.051983 28187 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:09.160342 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:09.159698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:43425 (local address 127.0.0.1:60642)
0809 05:01:09.159952 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:09.160082 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.193:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 05:01:09.160408 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 28095
2026-08-09T05:01:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2824 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:09 dist-test-slave-21b9 krb5kdc[28936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:09 dist-test-slave-21b9 krb5kdc[28936](info): set up 2 sockets
Aug 09 05:01:09 dist-test-slave-21b9 krb5kdc[28936](info): commencing operation
krb5kdc: starting...
W20260809 05:01:10.421525 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:10 dist-test-slave-21b9 krb5kdc[28936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 05:01:10.730906 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:40197
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:40197
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:11.068841 28953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:11.069151 28953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:11.069283 28953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:11.076368 28953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:11.076473 28953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:11.076545 28953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:11.076623 28953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:11.076675 28953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:11.086303 28953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:40197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:40197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:11.087670 28953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:11.088992 28953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:11.100308 28964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:11.100391 28963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:11.100922 28968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:11.101465 28953 server_base.cc:1034] running on GCE node
I20260809 05:01:11.102296 28953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:11.103817 28953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:11.105073 28953 hybrid_clock.cc:648] HybridClock initialized: now 1786251671105016 us; error 38 us; skew 500 ppm
I20260809 05:01:11.105446 28953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:11 dist-test-slave-21b9 krb5kdc[28936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251671, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:11.111701 28953 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:01:11.113274 28953 webserver.cc:460] Webserver started at http://127.29.243.254:39177/ using document root <none> and password file <none>
I20260809 05:01:11.114012 28953 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:11.114136 28953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:11.114437 28953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:11.117599 28953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "97e666a5cb0c44c2aeda1c864435464b"
format_stamp: "Formatted at 2026-08-09 05:01:11 on dist-test-slave-21b9"
server_key: "26fe0b014d4e0f59f7b90763a4990079"
server_key_iv: "678ca3606bd306d3f6a7b45782685943"
server_key_version: "encryptionkey@0"
I20260809 05:01:11.118336 28953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "97e666a5cb0c44c2aeda1c864435464b"
format_stamp: "Formatted at 2026-08-09 05:01:11 on dist-test-slave-21b9"
server_key: "26fe0b014d4e0f59f7b90763a4990079"
server_key_iv: "678ca3606bd306d3f6a7b45782685943"
server_key_version: "encryptionkey@0"
I20260809 05:01:11.123606 28953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:01:11.127058 28983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:11.128401 28953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:11.128567 28953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "97e666a5cb0c44c2aeda1c864435464b"
format_stamp: "Formatted at 2026-08-09 05:01:11 on dist-test-slave-21b9"
server_key: "26fe0b014d4e0f59f7b90763a4990079"
server_key_iv: "678ca3606bd306d3f6a7b45782685943"
server_key_version: "encryptionkey@0"
I20260809 05:01:11.128738 28953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:11.154417 28953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:11.158442 28953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:11.158701 28953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:11.176363 28953 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:40197
I20260809 05:01:11.176407 29064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:40197 every 8 connection(s)
I20260809 05:01:11.177675 28953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:11.179993 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 28953
I20260809 05:01:11.180191 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:11.180727 30671 external_mini_cluster.cc:1428] Setting key 0cd4212b67642573dd932d498eb32a53
I20260809 05:01:11.182245 29065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:01:11 dist-test-slave-21b9 krb5kdc[28936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:11.195921 29065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b: Bootstrap starting.
I20260809 05:01:11.199349 29065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:11.200611 29065 log.cc:824] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:11.203981 29065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b: No bootstrap required, opened a new log
I20260809 05:01:11.204113 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:11.182708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37456 (local address 127.29.243.254:40197)
0809 05:01:11.183494 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:11.183506 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:11.183546 (+ 40us) server_negotiation.cc:407] Connection header received
0809 05:01:11.185119 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:11.185140 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:11.185508 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:11.185828 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:11.187694 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:11.189021 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:11.190736 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:11.191427 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:11.191563 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:11.196412 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:11.196422 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:11.196441 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:11.196481 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:11.199602 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:11.200433 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:11.200440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:11.200446 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:11.200618 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:11.201099 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:11.201105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:11.201108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:11.201906 (+ 798us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:11.202186 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:11.202646 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 05:01:11.202979 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":332,"threads_started":1}
I20260809 05:01:11.211203 29065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e666a5cb0c44c2aeda1c864435464b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40197 } }
I20260809 05:01:11.211576 29065 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:11.211671 29065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e666a5cb0c44c2aeda1c864435464b, State: Initialized, Role: FOLLOWER
I20260809 05:01:11.212177 29065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e666a5cb0c44c2aeda1c864435464b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40197 } }
I20260809 05:01:11.212369 29065 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:11.212502 29065 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:11.212669 29065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:11.214733 29065 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e666a5cb0c44c2aeda1c864435464b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40197 } }
I20260809 05:01:11.215212 29065 leader_election.cc:302] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97e666a5cb0c44c2aeda1c864435464b; no voters:
I20260809 05:01:11.215831 29065 leader_election.cc:288] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:11.215956 29075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:11.216964 29075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [term 1 LEADER]: Becoming Leader. State: Replica: 97e666a5cb0c44c2aeda1c864435464b, State: Running, Role: LEADER
I20260809 05:01:11.217585 29075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e666a5cb0c44c2aeda1c864435464b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40197 } }
I20260809 05:01:11.217955 29065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:11.220315 29076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97e666a5cb0c44c2aeda1c864435464b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e666a5cb0c44c2aeda1c864435464b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40197 } } }
I20260809 05:01:11.220664 29076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:11.221333 29077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97e666a5cb0c44c2aeda1c864435464b. Latest consensus state: current_term: 1 leader_uuid: "97e666a5cb0c44c2aeda1c864435464b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e666a5cb0c44c2aeda1c864435464b" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40197 } } }
I20260809 05:01:11.221722 29077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:11.223096 29085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:11.228504 29085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:11.237325 29085 catalog_manager.cc:1347] Generated new cluster ID: 7550f27b4f1e479597d2d7c0541ecf70
I20260809 05:01:11.237452 29085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:11.253959 29085 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:11.255524 29085 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:11.264850 29085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b: Generated new TSK 0
I20260809 05:01:11.265722 29085 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.243.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
Entry for principal kudu/127.29.243.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab.
I20260809 05:01:11.338135 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:40197
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:11.688414 29114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:11.688694 29114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:11.688815 29114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:11.695915 29114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:11.696023 29114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:11.696170 29114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 05:01:11.705947 29114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.193.keytab
--principal=kudu/127.29.243.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:40197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:11.707343 29114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:11.708676 29114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:11.721913 29121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:11.721942 29123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:11.722477 29120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:11.723168 29114 server_base.cc:1034] running on GCE node
I20260809 05:01:11.723835 29114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:11.724865 29114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:11.726074 29114 hybrid_clock.cc:648] HybridClock initialized: now 1786251671726015 us; error 59 us; skew 500 ppm
I20260809 05:01:11.726405 29114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:11 dist-test-slave-21b9 krb5kdc[28936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251671, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:11.729625 29114 init.cc:377] Logged in from keytab as kudu/127.29.243.193@KRBTEST.COM (short username kudu)
I20260809 05:01:11.731112 29114 webserver.cc:460] Webserver started at http://127.29.243.193:38193/ using document root <none> and password file <none>
I20260809 05:01:11.731854 29114 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:11.732007 29114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:11.732340 29114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:11.735351 29114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "e4eff034a3b9414481ec0d6dd9bb32ca"
format_stamp: "Formatted at 2026-08-09 05:01:11 on dist-test-slave-21b9"
server_key: "ba4e668a9114f1e73780f14dfc913344"
server_key_iv: "56920dd576705e70b09356dcb19b4b19"
server_key_version: "encryptionkey@0"
I20260809 05:01:11.736124 29114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "e4eff034a3b9414481ec0d6dd9bb32ca"
format_stamp: "Formatted at 2026-08-09 05:01:11 on dist-test-slave-21b9"
server_key: "ba4e668a9114f1e73780f14dfc913344"
server_key_iv: "56920dd576705e70b09356dcb19b4b19"
server_key_version: "encryptionkey@0"
I20260809 05:01:11.741297 29114 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:01:11.744899 29135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:11.746227 29114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:01:11.746387 29114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "e4eff034a3b9414481ec0d6dd9bb32ca"
format_stamp: "Formatted at 2026-08-09 05:01:11 on dist-test-slave-21b9"
server_key: "ba4e668a9114f1e73780f14dfc913344"
server_key_iv: "56920dd576705e70b09356dcb19b4b19"
server_key_version: "encryptionkey@0"
I20260809 05:01:11.746570 29114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:11.768865 29114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:11.773083 29114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:11.773343 29114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:11.774384 29114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:11.775768 29114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:11.775846 29114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:11.775971 29114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:11.776027 29114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:11.807343 29114 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:37519
I20260809 05:01:11.807422 29320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:37519 every 8 connection(s)
I20260809 05:01:11.808723 29114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 05:01:11.819134 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 29114
I20260809 05:01:11.819337 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 05:01:11.819882 30671 external_mini_cluster.cc:1428] Setting key 90644ca0bb3edbcd1daadb67d6bb196e
Aug 09 05:01:11 dist-test-slave-21b9 krb5kdc[28936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251671, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.193@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:11.829306 29328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:11.811404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:54675 (local address 127.29.243.254:40197)
0809 05:01:11.811891 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:11.811897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:11.812819 (+ 922us) server_negotiation.cc:407] Connection header received
0809 05:01:11.814969 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:11.814974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:11.815043 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:11.815131 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:11.816580 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:11.817297 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:11.818963 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:11.819445 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:11.819544 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:11.823548 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:11.823555 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:11.823560 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:11.823597 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:11.826116 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:11.826927 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:11.826932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:11.826934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:11.827060 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:11.827573 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:11.827577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:11.827579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:11.827857 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:11.827981 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:11.828855 (+ 874us) server_negotiation.cc:299] Negotiation successful
0809 05:01:11.829033 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":211,"threads_started":1}
I20260809 05:01:11.830662 29329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:11.811905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:40197 (local address 127.29.243.193:54675)
0809 05:01:11.812656 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:11.812694 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:11.814695 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:11.815354 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:11.815362 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:11.815854 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:11.816413 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:11.816434 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:11.817437 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:11.817440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:11.818829 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:11.818844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:11.819669 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:11.819676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:11.819973 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:11.820660 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:11.820688 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:11.823194 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:11.826252 (+ 3058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:11.826261 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:11.826277 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:11.826818 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:11.827157 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:11.827161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:11.827164 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:11.827444 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:11.827985 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:11.827992 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:11.828421 (+ 429us) client_negotiation.cc:769] Sending connection context
0809 05:01:11.828729 (+ 308us) client_negotiation.cc:240] Negotiation successful
0809 05:01:11.829279 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":270,"threads_started":1}
I20260809 05:01:11.832459 29323 heartbeater.cc:347] Connected to a master server at 127.29.243.254:40197
I20260809 05:01:11.832803 29323 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:11.833628 29323 heartbeater.cc:511] Master 127.29.243.254:40197 requested a full tablet report, sending...
I20260809 05:01:11.836021 29013 ts_manager.cc:194] Registered new tserver with Master: e4eff034a3b9414481ec0d6dd9bb32ca (127.29.243.193:37519)
I20260809 05:01:11.837539 29013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.193@KRBTEST.COM'} at 127.29.243.193:54675
WARNING: no policy specified for kudu/127.29.243.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
Entry for principal kudu/127.29.243.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab.
I20260809 05:01:11.888635 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:40197
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:12.238473 29333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:12.238754 29333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:12.238849 29333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:12.246102 29333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:12.246209 29333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:12.246335 29333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 05:01:12.256062 29333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.194.keytab
--principal=kudu/127.29.243.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:40197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:12.257296 29333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:12.258598 29333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:12.271502 29341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:12.271535 29342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:12.271710 29344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:12.272207 29333 server_base.cc:1034] running on GCE node
I20260809 05:01:12.272799 29333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:12.273734 29333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:12.274940 29333 hybrid_clock.cc:648] HybridClock initialized: now 1786251672274884 us; error 46 us; skew 500 ppm
I20260809 05:01:12.275256 29333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:12 dist-test-slave-21b9 krb5kdc[28936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:12.278373 29333 init.cc:377] Logged in from keytab as kudu/127.29.243.194@KRBTEST.COM (short username kudu)
I20260809 05:01:12.279872 29333 webserver.cc:460] Webserver started at http://127.29.243.194:41413/ using document root <none> and password file <none>
I20260809 05:01:12.280527 29333 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:12.280627 29333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:12.280901 29333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:12.283813 29333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "261b718d59e347a893fed364e0e2a6c8"
format_stamp: "Formatted at 2026-08-09 05:01:12 on dist-test-slave-21b9"
server_key: "93225c7a1be97d77feda23e05793f6fb"
server_key_iv: "281c7d8d01db8ce5b3ed98a579a9c01a"
server_key_version: "encryptionkey@0"
I20260809 05:01:12.284548 29333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "261b718d59e347a893fed364e0e2a6c8"
format_stamp: "Formatted at 2026-08-09 05:01:12 on dist-test-slave-21b9"
server_key: "93225c7a1be97d77feda23e05793f6fb"
server_key_iv: "281c7d8d01db8ce5b3ed98a579a9c01a"
server_key_version: "encryptionkey@0"
I20260809 05:01:12.289850 29333 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:01:12.293493 29354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:12.295109 29333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:01:12.295248 29333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "261b718d59e347a893fed364e0e2a6c8"
format_stamp: "Formatted at 2026-08-09 05:01:12 on dist-test-slave-21b9"
server_key: "93225c7a1be97d77feda23e05793f6fb"
server_key_iv: "281c7d8d01db8ce5b3ed98a579a9c01a"
server_key_version: "encryptionkey@0"
I20260809 05:01:12.295471 29333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:12.315378 29333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:12.319945 29333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:12.320233 29333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:12.321200 29333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:12.322566 29333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:12.322644 29333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:12.322759 29333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:12.322808 29333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:12.351487 29333 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:42409
I20260809 05:01:12.351536 29526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:42409 every 8 connection(s)
I20260809 05:01:12.352762 29333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 05:01:12.359491 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 29333
I20260809 05:01:12.359687 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 05:01:12.360240 30671 external_mini_cluster.cc:1428] Setting key b908765031c3575dd4f009ca7db9dcd1
Aug 09 05:01:12 dist-test-slave-21b9 krb5kdc[28936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.194@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:12.373584 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:12.355365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:49917 (local address 127.29.243.254:40197)
0809 05:01:12.355800 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:12.355804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:12.356672 (+ 868us) server_negotiation.cc:407] Connection header received
0809 05:01:12.358561 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:12.358566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:12.358647 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:12.358744 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:12.360768 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.361738 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.363668 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.364111 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.364207 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:12.368097 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:12.368103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:12.368106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:12.368140 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:12.370621 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:12.371444 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:12.371448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:12.371450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:12.371584 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:12.372022 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:12.372025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:12.372026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:12.372262 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:12.372377 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:12.373152 (+ 775us) server_negotiation.cc:299] Negotiation successful
0809 05:01:12.373312 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":228,"threads_started":1}
I20260809 05:01:12.375052 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:12.355792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:40197 (local address 127.29.243.194:49917)
0809 05:01:12.356513 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:12.356552 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:12.358277 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:12.359008 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:12.359016 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:12.359769 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:12.360578 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:12.360606 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.361887 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.361891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:12.363532 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:12.363547 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.364336 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.364344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:12.364627 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:12.365311 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:12.365349 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:12.367852 (+ 2503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:12.370755 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:12.370764 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:12.370781 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:12.371346 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:12.371681 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:12.371686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:12.371689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:12.371919 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:12.372373 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:12.372380 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:12.372770 (+ 390us) client_negotiation.cc:769] Sending connection context
0809 05:01:12.373100 (+ 330us) client_negotiation.cc:240] Negotiation successful
0809 05:01:12.373591 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":269,"threads_started":1}
I20260809 05:01:12.376510 29528 heartbeater.cc:347] Connected to a master server at 127.29.243.254:40197
I20260809 05:01:12.376832 29528 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:12.377566 29528 heartbeater.cc:511] Master 127.29.243.254:40197 requested a full tablet report, sending...
I20260809 05:01:12.379117 29013 ts_manager.cc:194] Registered new tserver with Master: 261b718d59e347a893fed364e0e2a6c8 (127.29.243.194:42409)
I20260809 05:01:12.380213 29013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.194@KRBTEST.COM'} at 127.29.243.194:49917
WARNING: no policy specified for kudu/127.29.243.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
Entry for principal kudu/127.29.243.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab.
I20260809 05:01:12.429786 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:40197
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:12.769909 29534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:12.770179 29534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:12.770274 29534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:12.777263 29534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:12.777372 29534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:12.777498 29534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 05:01:12.787178 29534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.195.keytab
--principal=kudu/127.29.243.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:40197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:12.788499 29534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:12.789878 29534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:12.802933 29541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:12.802995 29544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:12.802973 29542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:12.804292 29534 server_base.cc:1034] running on GCE node
I20260809 05:01:12.804849 29534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:12.805716 29534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:12.806944 29534 hybrid_clock.cc:648] HybridClock initialized: now 1786251672806880 us; error 52 us; skew 500 ppm
I20260809 05:01:12.807333 29534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:12 dist-test-slave-21b9 krb5kdc[28936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:12.811215 29534 init.cc:377] Logged in from keytab as kudu/127.29.243.195@KRBTEST.COM (short username kudu)
I20260809 05:01:12.813472 29534 webserver.cc:460] Webserver started at http://127.29.243.195:43553/ using document root <none> and password file <none>
I20260809 05:01:12.814217 29534 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:12.814335 29534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:12.814648 29534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:12.817564 29534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "edd52837f9354bfab8544d620c44d073"
format_stamp: "Formatted at 2026-08-09 05:01:12 on dist-test-slave-21b9"
server_key: "d5232a7b9ac445941bf5eb853dbc128b"
server_key_iv: "dd44c35dbe935d746b0875ee0d3539cd"
server_key_version: "encryptionkey@0"
I20260809 05:01:12.818380 29534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "edd52837f9354bfab8544d620c44d073"
format_stamp: "Formatted at 2026-08-09 05:01:12 on dist-test-slave-21b9"
server_key: "d5232a7b9ac445941bf5eb853dbc128b"
server_key_iv: "dd44c35dbe935d746b0875ee0d3539cd"
server_key_version: "encryptionkey@0"
I20260809 05:01:12.823748 29534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 05:01:12.827324 29552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:12.828691 29534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:01:12.828850 29534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "edd52837f9354bfab8544d620c44d073"
format_stamp: "Formatted at 2026-08-09 05:01:12 on dist-test-slave-21b9"
server_key: "d5232a7b9ac445941bf5eb853dbc128b"
server_key_iv: "dd44c35dbe935d746b0875ee0d3539cd"
server_key_version: "encryptionkey@0"
I20260809 05:01:12.829027 29534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:12.840598 29323 heartbeater.cc:503] Master 127.29.243.254:40197 was elected leader, sending a full tablet report...
I20260809 05:01:12.848658 29534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:12.852757 29534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:12.853020 29534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:12.853987 29534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:12.855408 29534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:12.855484 29534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:12.855604 29534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:12.855654 29534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:12.886281 29534 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:41195
I20260809 05:01:12.886322 29737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:41195 every 8 connection(s)
I20260809 05:01:12.887620 29534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 05:01:12.889847 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 29534
I20260809 05:01:12.890069 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 05:01:12.890632 30671 external_mini_cluster.cc:1428] Setting key ff090051b0ee6fbe31dfc1af179638a1
Aug 09 05:01:12 dist-test-slave-21b9 krb5kdc[28936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.195@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:12.907264 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:12.890238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:38431 (local address 127.29.243.254:40197)
0809 05:01:12.890685 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:12.890691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:12.891905 (+ 1214us) server_negotiation.cc:407] Connection header received
0809 05:01:12.893856 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:12.893860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:12.893954 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:12.894045 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:12.895490 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.896255 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.897693 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.898091 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.898182 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:12.901593 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:12.901598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:12.901601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:12.901634 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:12.904301 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:12.904991 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:12.904995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:12.904998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:12.905118 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:12.905541 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:12.905545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:12.905547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:12.905834 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:12.905974 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:12.906659 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 05:01:12.906886 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":206,"threads_started":1}
I20260809 05:01:12.908243 29743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:12.890843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:40197 (local address 127.29.243.195:38431)
0809 05:01:12.891723 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:12.891786 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:12.893620 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:12.894236 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:12.894244 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:12.894741 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:12.895338 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:12.895357 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.896371 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.896374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:12.897562 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:12.897573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.898261 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.898267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:12.898492 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:12.899106 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:12.899133 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:12.901380 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:12.904438 (+ 3058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:12.904448 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:12.904461 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:12.904862 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:12.905223 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:12.905228 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:12.905231 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:12.905424 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:12.905962 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:12.905970 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:12.906279 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 05:01:12.906529 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 05:01:12.907040 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":343,"threads_started":1}
I20260809 05:01:12.909498 29738 heartbeater.cc:347] Connected to a master server at 127.29.243.254:40197
I20260809 05:01:12.909812 29738 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:12.910537 29738 heartbeater.cc:511] Master 127.29.243.254:40197 requested a full tablet report, sending...
I20260809 05:01:12.911964 29013 ts_manager.cc:194] Registered new tserver with Master: edd52837f9354bfab8544d620c44d073 (127.29.243.195:41195)
I20260809 05:01:12.912783 29013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.243.195@KRBTEST.COM'} at 127.29.243.195:38431
I20260809 05:01:12.917809 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:01:12.934583 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:12.922161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37460 (local address 127.29.243.254:40197)
0809 05:01:12.922352 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:12.922359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:12.922705 (+ 346us) server_negotiation.cc:407] Connection header received
0809 05:01:12.922884 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:12.922890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:12.922961 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:12.923051 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:12.924588 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.925306 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.926585 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.927021 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.927132 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:12.929385 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:12.929390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:12.929393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:12.929426 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:12.931886 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:12.932627 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:12.932633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:12.932636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:12.932777 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:12.933134 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:12.933140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:12.933143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:12.933512 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:12.933754 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:12.934074 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 05:01:12.934285 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 05:01:12.938263 29013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:01:12.945086 29013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:01:12.979562 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:12.957870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:42409 (local address 127.0.0.1:60016)
0809 05:01:12.970143 (+ 12273us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:12.970161 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:12.970401 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:12.971335 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:12.971339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:12.971360 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:12.971834 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:12.971846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.973792 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.973797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:12.976266 (+ 2469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:12.976283 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.978444 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.978450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:12.978633 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:12.978695 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 05:01:12.978812 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 05:01:12.979003 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12153,"mutex_wait_us":3,"thread_start_us":11192,"threads_started":1}
I20260809 05:01:12.980522 29764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:12.958429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.29.243.194:42409)
0809 05:01:12.967876 (+ 9447us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:12.967886 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:12.970277 (+ 2391us) server_negotiation.cc:407] Connection header received
0809 05:01:12.970595 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:12.970602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:12.970802 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:12.970977 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:12.972040 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.973623 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.976465 (+ 2842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.977926 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.978063 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:12.978251 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:12.979999 (+ 1748us) server_negotiation.cc:299] Negotiation successful
0809 05:01:12.980180 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9274,"thread_start_us":9156,"threads_started":1}
I20260809 05:01:12.980998 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:12.957433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:37519 (local address 127.0.0.1:35616)
0809 05:01:12.969460 (+ 12027us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:12.969510 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:12.969693 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:12.971051 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:12.971059 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:12.971096 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:12.971621 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:12.971639 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.974812 (+ 3173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.974817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:12.976934 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:12.976950 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.980184 (+ 3234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.980190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:12.980382 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:12.980414 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 05:01:12.980482 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 05:01:12.980605 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11775,"mutex_wait_us":22,"thread_start_us":12421,"threads_started":1}
I20260809 05:01:12.982714 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:12.957553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.29.243.193:37519)
0809 05:01:12.970391 (+ 12838us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:12.970398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:12.970425 (+ 27us) server_negotiation.cc:407] Connection header received
0809 05:01:12.970529 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:12.970537 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:12.970805 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:12.971843 (+ 1038us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:12.972118 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.974652 (+ 2534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.977137 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.978560 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.981958 (+ 3398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:12.982144 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:12.982232 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 05:01:12.982413 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12652,"thread_start_us":12479,"threads_started":1}
I20260809 05:01:12.985074 29423 tablet_service.cc:1467] Processing CreateTablet for tablet 3e91fc47fe1f412db1b469297c9317ea (DEFAULT_TABLE table=test-table [id=07ecb952e1d34a4389ed3c9858916a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:12.987202 29423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e91fc47fe1f412db1b469297c9317ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:12.986968 29216 tablet_service.cc:1467] Processing CreateTablet for tablet 3e91fc47fe1f412db1b469297c9317ea (DEFAULT_TABLE table=test-table [id=07ecb952e1d34a4389ed3c9858916a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:12.988201 29216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e91fc47fe1f412db1b469297c9317ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:12.990610 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:12.970487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41195 (local address 127.0.0.1:55916)
0809 05:01:12.981192 (+ 10705us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:12.981209 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:12.981309 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:12.984208 (+ 2899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:12.984213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:12.984241 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:12.984403 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:12.984413 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.986236 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.986241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:12.988046 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:12.988071 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.989844 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.989851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:12.990053 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:12.990089 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 05:01:12.990188 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 05:01:12.990339 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10607,"thread_start_us":13181,"threads_started":1}
I20260809 05:01:12.990747 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:12.970502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55916 (local address 127.29.243.195:41195)
0809 05:01:12.983605 (+ 13103us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:12.983614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:12.983649 (+ 35us) server_negotiation.cc:407] Connection header received
0809 05:01:12.983748 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:12.983755 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:12.983985 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:12.984210 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:12.984558 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.986058 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.988309 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:12.989669 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:12.989863 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:12.990025 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:12.990250 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 05:01:12.990418 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12929,"thread_start_us":12753,"threads_started":1}
I20260809 05:01:12.994580 29628 tablet_service.cc:1467] Processing CreateTablet for tablet 3e91fc47fe1f412db1b469297c9317ea (DEFAULT_TABLE table=test-table [id=07ecb952e1d34a4389ed3c9858916a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:12.996122 29628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e91fc47fe1f412db1b469297c9317ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:12.999925 29768 tablet_bootstrap.cc:492] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: Bootstrap starting.
I20260809 05:01:13.001852 29769 tablet_bootstrap.cc:492] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: Bootstrap starting.
I20260809 05:01:13.003823 29768 tablet_bootstrap.cc:654] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:13.005129 29768 log.cc:824] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:13.005298 29769 tablet_bootstrap.cc:654] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:13.006572 29769 log.cc:824] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:13.009243 29768 tablet_bootstrap.cc:492] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: No bootstrap required, opened a new log
I20260809 05:01:13.009392 29774 tablet_bootstrap.cc:492] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: Bootstrap starting.
I20260809 05:01:13.009549 29768 ts_tablet_manager.cc:1397] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260809 05:01:13.009888 29769 tablet_bootstrap.cc:492] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: No bootstrap required, opened a new log
I20260809 05:01:13.010186 29769 ts_tablet_manager.cc:1397] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 05:01:13.013232 29774 tablet_bootstrap.cc:654] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:13.014458 29774 log.cc:824] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:13.017524 29774 tablet_bootstrap.cc:492] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: No bootstrap required, opened a new log
I20260809 05:01:13.017412 29769 raft_consensus.cc:348] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } }
I20260809 05:01:13.017796 29774 ts_tablet_manager.cc:1397] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 05:01:13.017802 29769 raft_consensus.cc:374] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:13.017912 29769 raft_consensus.cc:729] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 261b718d59e347a893fed364e0e2a6c8, State: Initialized, Role: FOLLOWER
I20260809 05:01:13.018749 29769 consensus_queue.cc:260] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } }
I20260809 05:01:13.019198 29768 raft_consensus.cc:348] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } }
I20260809 05:01:13.019630 29768 raft_consensus.cc:374] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:13.019722 29768 raft_consensus.cc:729] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4eff034a3b9414481ec0d6dd9bb32ca, State: Initialized, Role: FOLLOWER
I20260809 05:01:13.020308 29768 consensus_queue.cc:260] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } }
I20260809 05:01:13.022070 29768 ts_tablet_manager.cc:1428] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260809 05:01:13.027467 29528 heartbeater.cc:503] Master 127.29.243.254:40197 was elected leader, sending a full tablet report...
I20260809 05:01:13.027387 29774 raft_consensus.cc:348] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } }
I20260809 05:01:13.027858 29774 raft_consensus.cc:374] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:13.027957 29774 raft_consensus.cc:729] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edd52837f9354bfab8544d620c44d073, State: Initialized, Role: FOLLOWER
I20260809 05:01:13.028474 29769 ts_tablet_manager.cc:1428] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: Time spent starting tablet: real 0.018s user 0.006s sys 0.009s
I20260809 05:01:13.028488 29774 consensus_queue.cc:260] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } }
I20260809 05:01:13.029471 29738 heartbeater.cc:503] Master 127.29.243.254:40197 was elected leader, sending a full tablet report...
I20260809 05:01:13.030153 29774 ts_tablet_manager.cc:1428] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260809 05:01:13.061086 29325 tablet.cc:2369] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:01:13.104737 29529 tablet.cc:2369] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:01:13.139240 29739 tablet.cc:2369] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:01:13.446738 29777 raft_consensus.cc:482] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:01:13.447185 29779 raft_consensus.cc:482] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:01:13.447211 29777 raft_consensus.cc:504] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } }
I20260809 05:01:13.447484 29779 raft_consensus.cc:504] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } }
I20260809 05:01:13.447942 29776 raft_consensus.cc:482] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:01:13.448248 29776 raft_consensus.cc:504] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } }
I20260809 05:01:13.449528 29777 leader_election.cc:288] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edd52837f9354bfab8544d620c44d073 (127.29.243.195:41195), 261b718d59e347a893fed364e0e2a6c8 (127.29.243.194:42409)
I20260809 05:01:13.451005 29776 leader_election.cc:288] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4eff034a3b9414481ec0d6dd9bb32ca (127.29.243.193:37519), edd52837f9354bfab8544d620c44d073 (127.29.243.195:41195)
I20260809 05:01:13.449446 29779 leader_election.cc:288] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4eff034a3b9414481ec0d6dd9bb32ca (127.29.243.193:37519), 261b718d59e347a893fed364e0e2a6c8 (127.29.243.194:42409)
I20260809 05:01:13.461910 29781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.449972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:42409 (local address 127.29.243.193:52011)
0809 05:01:13.451830 (+ 1858us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:13.451853 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:13.452036 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:13.452847 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:13.452852 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:13.452896 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:13.453425 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.453444 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.456454 (+ 3010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.456460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.458760 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.458777 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.460879 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.460886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.461135 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.461177 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 05:01:13.461284 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 05:01:13.461438 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1702,"mutex_wait_us":74,"thread_start_us":1465,"threads_started":1}
I20260809 05:01:13.462815 29782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.450139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:52011 (local address 127.29.243.194:42409)
0809 05:01:13.452282 (+ 2143us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:13.452287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:13.452313 (+ 26us) server_negotiation.cc:407] Connection header received
0809 05:01:13.452385 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:13.452390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:13.452459 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:13.452563 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:13.453652 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.456248 (+ 2596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.458963 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.460412 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.460615 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.460712 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:13.462240 (+ 1528us) server_negotiation.cc:299] Negotiation successful
0809 05:01:13.462411 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2021,"thread_start_us":1872,"threads_started":1}
I20260809 05:01:13.463240 29764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.450026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:45137 (local address 127.29.243.194:42409)
0809 05:01:13.450428 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:13.450433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:13.454672 (+ 4239us) server_negotiation.cc:407] Connection header received
0809 05:01:13.455033 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:13.455039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:13.455118 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:13.455216 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:13.456904 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.458141 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.460698 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.461987 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.462129 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.462208 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:13.462791 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 05:01:13.462935 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274}
I20260809 05:01:13.463909 29455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fc47fe1f412db1b469297c9317ea" candidate_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "261b718d59e347a893fed364e0e2a6c8" is_pre_election: true
I20260809 05:01:13.464457 29455 raft_consensus.cc:2454] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4eff034a3b9414481ec0d6dd9bb32ca in term 0.
I20260809 05:01:13.465356 29137 leader_election.cc:302] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 261b718d59e347a893fed364e0e2a6c8, e4eff034a3b9414481ec0d6dd9bb32ca; no voters:
I20260809 05:01:13.465925 29777 raft_consensus.cc:2781] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:01:13.466096 29777 raft_consensus.cc:482] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:01:13.466214 29777 raft_consensus.cc:3037] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:13.466325 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.454568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:37235 (local address 127.29.243.195:41195)
0809 05:01:13.454726 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:13.454733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:13.454748 (+ 15us) server_negotiation.cc:407] Connection header received
0809 05:01:13.454802 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:13.454806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:13.454871 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:13.454964 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:13.455665 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.456827 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.463833 (+ 7006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.465105 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.465238 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.465314 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:13.465914 (+ 600us) server_negotiation.cc:299] Negotiation successful
0809 05:01:13.466055 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":58}
I20260809 05:01:13.466377 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.451714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:37519 (local address 127.29.243.194:59979)
0809 05:01:13.456016 (+ 4302us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:13.456033 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:13.456148 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:13.456727 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:13.456732 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:13.456756 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:13.457268 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.457282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.461237 (+ 3955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.461244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.463604 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.463621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.465508 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.465516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.465698 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.465735 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:01:13.465839 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 05:01:13.465975 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4172,"thread_start_us":5210,"threads_started":1}
I20260809 05:01:13.467698 29794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.452116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:59979 (local address 127.29.243.193:37519)
0809 05:01:13.452822 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:13.452828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:13.456261 (+ 3433us) server_negotiation.cc:407] Connection header received
0809 05:01:13.456317 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:13.456321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:13.456381 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:13.456462 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:13.457515 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.458862 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.463791 (+ 4929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.465037 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.465234 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.465328 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:13.466918 (+ 1590us) server_negotiation.cc:299] Negotiation successful
0809 05:01:13.467214 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":567,"thread_start_us":484,"threads_started":1}
I20260809 05:01:13.467739 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.450852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41195 (local address 127.29.243.193:37235)
0809 05:01:13.453635 (+ 2783us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:13.453653 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:13.453764 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:13.455084 (+ 1320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:13.455089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:13.455114 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:13.455498 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.455512 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.459020 (+ 3508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.459025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.461058 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.461074 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.465534 (+ 4460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.465540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.465709 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.465743 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 05:01:13.466618 (+ 875us) client_negotiation.cc:240] Negotiation successful
0809 05:01:13.467186 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2643,"mutex_wait_us":3,"thread_start_us":2492,"threads_started":1}
I20260809 05:01:13.469020 29666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fc47fe1f412db1b469297c9317ea" candidate_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edd52837f9354bfab8544d620c44d073" is_pre_election: true
I20260809 05:01:13.469609 29666 raft_consensus.cc:2454] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4eff034a3b9414481ec0d6dd9bb32ca in term 0.
I20260809 05:01:13.469874 29777 raft_consensus.cc:504] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } }
I20260809 05:01:13.471158 29666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fc47fe1f412db1b469297c9317ea" candidate_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edd52837f9354bfab8544d620c44d073"
I20260809 05:01:13.471302 29455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fc47fe1f412db1b469297c9317ea" candidate_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "261b718d59e347a893fed364e0e2a6c8"
I20260809 05:01:13.471395 29666 raft_consensus.cc:3037] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:13.471537 29455 raft_consensus.cc:3037] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:13.474478 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.457665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:59075 (local address 127.29.243.195:41195)
0809 05:01:13.466573 (+ 8908us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:13.466578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:13.466675 (+ 97us) server_negotiation.cc:407] Connection header received
0809 05:01:13.466863 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:13.466882 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:13.466956 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:13.467042 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:13.468394 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.469619 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.471947 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.473344 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.473518 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.473586 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:13.473979 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 05:01:13.474118 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":8802,"threads_started":1}
I20260809 05:01:13.474867 29666 raft_consensus.cc:2454] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4eff034a3b9414481ec0d6dd9bb32ca in term 1.
I20260809 05:01:13.474886 29455 raft_consensus.cc:2454] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4eff034a3b9414481ec0d6dd9bb32ca in term 1.
I20260809 05:01:13.475097 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.457357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:41195 (local address 127.29.243.194:59075)
0809 05:01:13.466592 (+ 9235us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:13.466614 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:13.466743 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:13.467980 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:13.467984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:13.468016 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:13.468187 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.468199 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.469872 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.469878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.471712 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.471749 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.473515 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.473522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.473757 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.473800 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 05:01:13.474618 (+ 818us) client_negotiation.cc:240] Negotiation successful
0809 05:01:13.474768 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9118,"threads_started":1}
I20260809 05:01:13.475414 29777 leader_election.cc:288] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [CANDIDATE]: Term 1 election: Requested vote from peers edd52837f9354bfab8544d620c44d073 (127.29.243.195:41195), 261b718d59e347a893fed364e0e2a6c8 (127.29.243.194:42409)
I20260809 05:01:13.475414 29137 leader_election.cc:302] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 261b718d59e347a893fed364e0e2a6c8, e4eff034a3b9414481ec0d6dd9bb32ca; no voters:
I20260809 05:01:13.475780 29777 raft_consensus.cc:2781] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:13.476217 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.450156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:42409 (local address 127.29.243.195:45137)
0809 05:01:13.454582 (+ 4426us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:13.454598 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:13.454909 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:13.456143 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:13.456148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:13.456178 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:13.456665 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.456679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.458348 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.458354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.460497 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.460514 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.462353 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.462360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.462575 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.462617 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 05:01:13.475628 (+ 13011us) client_negotiation.cc:240] Negotiation successful
0809 05:01:13.475787 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4309,"thread_start_us":2273,"threads_started":1}
I20260809 05:01:13.477173 29455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fc47fe1f412db1b469297c9317ea" candidate_uuid: "edd52837f9354bfab8544d620c44d073" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "261b718d59e347a893fed364e0e2a6c8" is_pre_election: true
I20260809 05:01:13.477489 29455 raft_consensus.cc:2379] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate edd52837f9354bfab8544d620c44d073 in current term 1: Already voted for candidate e4eff034a3b9414481ec0d6dd9bb32ca in this term.
I20260809 05:01:13.477901 29666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fc47fe1f412db1b469297c9317ea" candidate_uuid: "261b718d59e347a893fed364e0e2a6c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edd52837f9354bfab8544d620c44d073" is_pre_election: true
I20260809 05:01:13.477993 29248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fc47fe1f412db1b469297c9317ea" candidate_uuid: "261b718d59e347a893fed364e0e2a6c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" is_pre_election: true
I20260809 05:01:13.478343 29666 raft_consensus.cc:2379] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 261b718d59e347a893fed364e0e2a6c8 in current term 1: Already voted for candidate e4eff034a3b9414481ec0d6dd9bb32ca in this term.
I20260809 05:01:13.478797 29777 raft_consensus.cc:686] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 1 LEADER]: Becoming Leader. State: Replica: e4eff034a3b9414481ec0d6dd9bb32ca, State: Running, Role: LEADER
I20260809 05:01:13.479408 29777 consensus_queue.cc:237] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } }
I20260809 05:01:13.479780 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.449572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:37519 (local address 127.29.243.195:42607)
0809 05:01:13.452435 (+ 2863us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:13.452456 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:13.452675 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:13.453245 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:13.453250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:13.453280 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:13.453610 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.453625 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.454977 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.454982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.476014 (+ 21032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:13.476032 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.479019 (+ 2987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.479027 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:13.479198 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.479226 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 05:01:13.479345 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 05:01:13.479457 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2725,"mutex_wait_us":16060,"thread_start_us":4709,"threads_started":1}
I20260809 05:01:13.479836 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.450177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:42607 (local address 127.29.243.193:37519)
0809 05:01:13.450417 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:13.450425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:13.452548 (+ 2123us) server_negotiation.cc:407] Connection header received
0809 05:01:13.452867 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:13.452873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:13.452942 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:13.453004 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:13.453792 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.454743 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.476225 (+ 21482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.477465 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.477585 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.477651 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:13.479378 (+ 1727us) server_negotiation.cc:299] Negotiation successful
0809 05:01:13.479554 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":72}
I20260809 05:01:13.480433 29247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fc47fe1f412db1b469297c9317ea" candidate_uuid: "edd52837f9354bfab8544d620c44d073" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" is_pre_election: true
I20260809 05:01:13.481454 29558 leader_election.cc:302] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: edd52837f9354bfab8544d620c44d073; no voters: 261b718d59e347a893fed364e0e2a6c8, e4eff034a3b9414481ec0d6dd9bb32ca
I20260809 05:01:13.481827 29779 raft_consensus.cc:2726] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 05:01:13.483608 29359 leader_election.cc:302] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 261b718d59e347a893fed364e0e2a6c8; no voters: e4eff034a3b9414481ec0d6dd9bb32ca, edd52837f9354bfab8544d620c44d073
I20260809 05:01:13.483963 29776 raft_consensus.cc:2726] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 05:01:13.485157 29010 catalog_manager.cc:5495] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca reported cstate change: term changed from 0 to 1, leader changed from <none> to e4eff034a3b9414481ec0d6dd9bb32ca (127.29.243.193). New cstate: current_term: 1 leader_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4eff034a3b9414481ec0d6dd9bb32ca" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 37519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 } health_report { overall_health: UNKNOWN } } }
I20260809 05:01:13.532612 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.526581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35630 (local address 127.29.243.193:37519)
0809 05:01:13.526750 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:13.526755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:13.527211 (+ 456us) server_negotiation.cc:407] Connection header received
0809 05:01:13.527458 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:13.527462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:13.527575 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:13.527708 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:01:13.528126 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.528871 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.530329 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.530752 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.530846 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.531394 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:01:13.531966 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:01:13.532099 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:13.532277 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 05:01:13.532358 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 05:01:13.540335 29455 raft_consensus.cc:1261] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 1 FOLLOWER]: Refusing update from remote peer e4eff034a3b9414481ec0d6dd9bb32ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:13.540366 29666 raft_consensus.cc:1261] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 1 FOLLOWER]: Refusing update from remote peer e4eff034a3b9414481ec0d6dd9bb32ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:13.541146 29800 consensus_queue.cc:1035] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "edd52837f9354bfab8544d620c44d073" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 41195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:01:13.541517 29777 consensus_queue.cc:1035] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "261b718d59e347a893fed364e0e2a6c8" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:01:13.567724 29013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:01:13.568042 29013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:01:13.571753 29013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97e666a5cb0c44c2aeda1c864435464b: Sending DeleteTablet for 3 replicas of tablet 3e91fc47fe1f412db1b469297c9317ea
I20260809 05:01:13.573050 29216 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e91fc47fe1f412db1b469297c9317ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:13 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:35616
I20260809 05:01:13.573100 29423 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e91fc47fe1f412db1b469297c9317ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:13 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:60016
I20260809 05:01:13.573932 29826 tablet_replica.cc:330] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: stopping tablet replica
I20260809 05:01:13.573927 29628 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e91fc47fe1f412db1b469297c9317ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:13 UTC) from {username='kudu', principal='kudu/127.29.243.254@KRBTEST.COM'} at 127.0.0.1:55916
I20260809 05:01:13.574512 29826 raft_consensus.cc:2229] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:01:13.575230 29826 raft_consensus.cc:2258] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:13.575601 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 29114
I20260809 05:01:13.577422 29826 ts_tablet_manager.cc:1905] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:13.585448 29826 ts_tablet_manager.cc:1918] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:13.586155 29829 tablet_replica.cc:330] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: stopping tablet replica
I20260809 05:01:13.586755 29829 raft_consensus.cc:2229] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:01:13.587445 29829 raft_consensus.cc:2258] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:13.589746 29829 ts_tablet_manager.cc:1905] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:13.596807 29829 ts_tablet_manager.cc:1918] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:13.597069 29826 log.cc:1196] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/3e91fc47fe1f412db1b469297c9317ea
I20260809 05:01:13.597712 29826 ts_tablet_manager.cc:1939] T 3e91fc47fe1f412db1b469297c9317ea P e4eff034a3b9414481ec0d6dd9bb32ca: Deleting consensus metadata
I20260809 05:01:13.599658 29829 log.cc:1196] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/3e91fc47fe1f412db1b469297c9317ea
I20260809 05:01:13.600584 28990 catalog_manager.cc:4841] TS e4eff034a3b9414481ec0d6dd9bb32ca (127.29.243.193:37519): tablet 3e91fc47fe1f412db1b469297c9317ea (table test-table [id=07ecb952e1d34a4389ed3c9858916a69]) successfully deleted
I20260809 05:01:13.602674 29829 ts_tablet_manager.cc:1939] T 3e91fc47fe1f412db1b469297c9317ea P edd52837f9354bfab8544d620c44d073: Deleting consensus metadata
Aug 09 05:01:13 dist-test-slave-21b9 krb5kdc[28936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.193@KRBTEST.COM
I20260809 05:01:13.605059 29828 tablet_replica.cc:330] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: stopping tablet replica
I20260809 05:01:13.605165 28990 catalog_manager.cc:4841] TS edd52837f9354bfab8544d620c44d073 (127.29.243.195:41195): tablet 3e91fc47fe1f412db1b469297c9317ea (table test-table [id=07ecb952e1d34a4389ed3c9858916a69]) successfully deleted
I20260809 05:01:13.605543 29828 raft_consensus.cc:2229] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:01:13.606101 29828 raft_consensus.cc:2258] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:13.608110 29828 ts_tablet_manager.cc:1905] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:13.612051 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.577457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.29.243.193:37519)
0809 05:01:13.577649 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:13.577655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:13.592321 (+ 14666us) server_negotiation.cc:407] Connection header received
0809 05:01:13.592811 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:13.592817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:13.592924 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:13.592996 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:13.594781 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.595969 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.597730 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.598426 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.598523 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.604211 (+ 5688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:13.604226 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:13.604245 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:13.604288 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:13.608695 (+ 4407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:13.609615 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:13.609626 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:13.609630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:13.609804 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:13.610284 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:13.610289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:13.610293 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:13.610786 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:13.611118 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:13.611466 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 05:01:13.611689 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:01:13.612779 29314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:13.614715 29828 ts_tablet_manager.cc:1918] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:13.614877 29828 log.cc:1196] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/3e91fc47fe1f412db1b469297c9317ea
I20260809 05:01:13.615442 29828 ts_tablet_manager.cc:1939] T 3e91fc47fe1f412db1b469297c9317ea P 261b718d59e347a893fed364e0e2a6c8: Deleting consensus metadata
I20260809 05:01:13.617125 28988 catalog_manager.cc:4841] TS 261b718d59e347a893fed364e0e2a6c8 (127.29.243.194:42409): tablet 3e91fc47fe1f412db1b469297c9317ea (table test-table [id=07ecb952e1d34a4389ed3c9858916a69]) successfully deleted
I20260809 05:01:13.712085 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 29114
I20260809 05:01:13.741549 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 29333
Aug 09 05:01:13 dist-test-slave-21b9 krb5kdc[28936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.194@KRBTEST.COM
I20260809 05:01:13.755584 29782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.742284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60030 (local address 127.29.243.194:42409)
0809 05:01:13.742522 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:13.742528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:13.742546 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:01:13.742711 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:13.742715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:13.742777 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:13.742866 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:13.744150 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.744845 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.746099 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.746470 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.746568 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.750527 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:13.750541 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:13.750559 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:13.750592 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:13.753321 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:13.753867 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:13.753873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:13.753876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:13.754013 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:13.754300 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:13.754303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:13.754305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:13.754689 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:13.754934 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:13.755099 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 05:01:13.755302 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 05:01:13.756239 29518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:13.845798 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 29333
I20260809 05:01:13.872601 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 29534
Aug 09 05:01:13 dist-test-slave-21b9 krb5kdc[28936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.195@KRBTEST.COM
I20260809 05:01:13.887874 29797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:13.873384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55926 (local address 127.29.243.195:41195)
0809 05:01:13.873612 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:13.873618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:13.873649 (+ 31us) server_negotiation.cc:407] Connection header received
0809 05:01:13.873808 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:13.873812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:13.873887 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:13.873984 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:13.875346 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.876088 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.877357 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:13.877777 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:13.877889 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:13.882398 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:13.882416 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:13.882435 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:13.882473 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:13.885080 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:13.885754 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:13.885762 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:13.885765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:13.885936 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:13.886295 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:13.886298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:13.886301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:13.886667 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:13.886960 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:13.887258 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 05:01:13.887518 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 05:01:13.888638 29730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:13.978636 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 29534
I20260809 05:01:14.005997 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 28953
I20260809 05:01:14.007172 29059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:14.132447 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 28953
2026-08-09T05:01:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4973 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T05:01:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:14Z Disabled control of system clock
I20260809 05:01:14.232661 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:45775
--webserver_interface=127.29.243.254
--webserver_port=0
--builtin_ntp_servers=127.29.243.212:35651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:45775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:01:14.575147 29840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:14.575497 29840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:14.575588 29840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:14.583153 29840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:14.583309 29840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:14.583382 29840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:14.583467 29840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:14.583534 29840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:14.593195 29840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:35651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:45775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:45775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:14.594530 29840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:14.596436 29840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:14.607689 29849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:14.607745 29853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:14.607718 29851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:14.608812 29840 server_base.cc:1034] running on GCE node
I20260809 05:01:14.609599 29840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:14.611042 29840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:14.612222 29840 hybrid_clock.cc:648] HybridClock initialized: now 1786251674612164 us; error 54 us; skew 500 ppm
I20260809 05:01:14.612568 29840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:01:14.614748 29840 webserver.cc:460] Webserver started at http://127.29.243.254:45217/ using document root <none> and password file <none>
I20260809 05:01:14.615468 29840 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:14.615562 29840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:14.615851 29840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:14.618685 29840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "6b0ea6d1941844c2937c060a740d10ab"
format_stamp: "Formatted at 2026-08-09 05:01:14 on dist-test-slave-21b9"
server_key: "29e7bb130a21ae17dacd3e83d8951a23"
server_key_iv: "9063538cb8c7c6119c82a8784ed58339"
server_key_version: "encryptionkey@0"
I20260809 05:01:14.619506 29840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "6b0ea6d1941844c2937c060a740d10ab"
format_stamp: "Formatted at 2026-08-09 05:01:14 on dist-test-slave-21b9"
server_key: "29e7bb130a21ae17dacd3e83d8951a23"
server_key_iv: "9063538cb8c7c6119c82a8784ed58339"
server_key_version: "encryptionkey@0"
I20260809 05:01:14.624704 29840 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 05:01:14.628544 29863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:14.630108 29840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 05:01:14.630313 29840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "6b0ea6d1941844c2937c060a740d10ab"
format_stamp: "Formatted at 2026-08-09 05:01:14 on dist-test-slave-21b9"
server_key: "29e7bb130a21ae17dacd3e83d8951a23"
server_key_iv: "9063538cb8c7c6119c82a8784ed58339"
server_key_version: "encryptionkey@0"
I20260809 05:01:14.630596 29840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:14.645355 29840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:14.646224 29840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:14.646510 29840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:14.664407 29840 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:45775
I20260809 05:01:14.664427 29947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:45775 every 8 connection(s)
I20260809 05:01:14.665766 29840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:14.669680 29949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:14.671519 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 29840
I20260809 05:01:14.671695 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:14.672151 30671 external_mini_cluster.cc:1428] Setting key 03cd9139200b843df0e714a9f2bf3009
I20260809 05:01:14.681197 29949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab: Bootstrap starting.
I20260809 05:01:14.684515 29949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:14.685627 29949 log.cc:824] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:14.685892 29954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:14.674112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54360 (local address 127.29.243.254:45775)
0809 05:01:14.675185 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:14.675195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:14.675241 (+ 46us) server_negotiation.cc:407] Connection header received
0809 05:01:14.677036 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:14.677058 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:14.677522 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:14.677867 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:14.678457 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:14.679836 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:14.681212 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:14.681959 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:14.682153 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:14.683372 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:14.683382 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:14.683396 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:14.683966 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:14.684067 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:14.684091 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:14.684200 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:14.684439 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 05:01:14.684795 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":391,"threads_started":1}
I20260809 05:01:14.688669 29949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab: No bootstrap required, opened a new log
I20260809 05:01:14.696201 29949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ea6d1941844c2937c060a740d10ab" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 45775 } }
I20260809 05:01:14.696542 29949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:14.696624 29949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0ea6d1941844c2937c060a740d10ab, State: Initialized, Role: FOLLOWER
I20260809 05:01:14.697114 29949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ea6d1941844c2937c060a740d10ab" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 45775 } }
I20260809 05:01:14.697315 29949 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:14.697417 29949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:14.697546 29949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:14.699613 29949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ea6d1941844c2937c060a740d10ab" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 45775 } }
I20260809 05:01:14.700050 29949 leader_election.cc:302] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b0ea6d1941844c2937c060a740d10ab; no voters:
I20260809 05:01:14.700670 29949 leader_election.cc:288] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:14.700816 29957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:14.702589 29949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:14.702885 29957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [term 1 LEADER]: Becoming Leader. State: Replica: 6b0ea6d1941844c2937c060a740d10ab, State: Running, Role: LEADER
I20260809 05:01:14.703385 29957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ea6d1941844c2937c060a740d10ab" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 45775 } }
I20260809 05:01:14.706418 29959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b0ea6d1941844c2937c060a740d10ab. Latest consensus state: current_term: 1 leader_uuid: "6b0ea6d1941844c2937c060a740d10ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ea6d1941844c2937c060a740d10ab" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 45775 } } }
I20260809 05:01:14.706446 29958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b0ea6d1941844c2937c060a740d10ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ea6d1941844c2937c060a740d10ab" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 45775 } } }
I20260809 05:01:14.706852 29959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:14.706856 29958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:14.708303 29972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:14.713641 29972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:14.721987 29972 catalog_manager.cc:1347] Generated new cluster ID: e33ccf850eec435e8191367a87a93ed2
I20260809 05:01:14.722092 29972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:14.753696 29972 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:14.754901 29972 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:14.765295 29972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab: Generated new TSK 0
I20260809 05:01:14.766078 29972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:14.782747 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.193:0
--local_ip_for_outbound_sockets=127.29.243.193
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:45775
--builtin_ntp_servers=127.29.243.212:35651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:01:15.128588 29984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:15.128870 29984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:15.129006 29984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:15.136253 29984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:15.136351 29984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:15.136492 29984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.193
I20260809 05:01:15.145450 29984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:35651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.243.193
--webserver_port=0
--tserver_master_addrs=127.29.243.254:45775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.193
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:15.146739 29984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:15.148627 29984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:15.161661 29997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:15.161691 29998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:15.161715 30001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:15.162708 29984 server_base.cc:1034] running on GCE node
I20260809 05:01:15.163344 29984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:15.164294 29984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:15.165501 29984 hybrid_clock.cc:648] HybridClock initialized: now 1786251675165440 us; error 53 us; skew 500 ppm
I20260809 05:01:15.165815 29984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:01:15.168000 29984 webserver.cc:460] Webserver started at http://127.29.243.193:34511/ using document root <none> and password file <none>
I20260809 05:01:15.168685 29984 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:15.168788 29984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:15.169114 29984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:15.171986 29984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/data/instance:
uuid: "9e65fdb3d6224dc9b3a71974d61a54cb"
format_stamp: "Formatted at 2026-08-09 05:01:15 on dist-test-slave-21b9"
server_key: "d6f2d9ab171ee318e26ad7e0827865a9"
server_key_iv: "921b3653e790c3d944c42e655117f03b"
server_key_version: "encryptionkey@0"
I20260809 05:01:15.172751 29984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance:
uuid: "9e65fdb3d6224dc9b3a71974d61a54cb"
format_stamp: "Formatted at 2026-08-09 05:01:15 on dist-test-slave-21b9"
server_key: "d6f2d9ab171ee318e26ad7e0827865a9"
server_key_iv: "921b3653e790c3d944c42e655117f03b"
server_key_version: "encryptionkey@0"
I20260809 05:01:15.177731 29984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:01:15.181097 30007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:15.182319 29984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:01:15.182478 29984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/wal
uuid: "9e65fdb3d6224dc9b3a71974d61a54cb"
format_stamp: "Formatted at 2026-08-09 05:01:15 on dist-test-slave-21b9"
server_key: "d6f2d9ab171ee318e26ad7e0827865a9"
server_key_iv: "921b3653e790c3d944c42e655117f03b"
server_key_version: "encryptionkey@0"
I20260809 05:01:15.182675 29984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:15.208374 29984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:15.209174 29984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:15.209457 29984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:15.210412 29984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:15.211849 29984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:15.211959 29984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:15.212066 29984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:15.212136 29984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:15.242453 29984 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.193:34561
I20260809 05:01:15.242509 30196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.193:34561 every 8 connection(s)
I20260809 05:01:15.243765 29984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/data/info.pb
I20260809 05:01:15.245209 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 29984
I20260809 05:01:15.245374 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/wal/instance
I20260809 05:01:15.245869 30671 external_mini_cluster.cc:1428] Setting key fcd8f3813d34c932c840fdcaa8524f83
I20260809 05:01:15.248843 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.194:0
--local_ip_for_outbound_sockets=127.29.243.194
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:45775
--builtin_ntp_servers=127.29.243.212:35651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:01:15.256307 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:15.246324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.193:48273 (local address 127.29.243.254:45775)
0809 05:01:15.246834 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:15.246841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:15.248251 (+ 1410us) server_negotiation.cc:407] Connection header received
0809 05:01:15.250165 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:15.250171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:15.250254 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:15.250348 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:15.251456 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:15.252179 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:15.253674 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:15.254071 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:15.254182 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:15.255480 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:15.255486 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:15.255489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:15.255506 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:15.255549 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:15.255561 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:15.255631 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:15.255855 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 05:01:15.255957 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":272,"threads_started":1}
I20260809 05:01:15.257236 30204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:15.247026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:45775 (local address 127.29.243.193:48273)
0809 05:01:15.248091 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:15.248138 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:15.249904 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:15.250600 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:15.250608 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:15.250693 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:15.251260 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:15.251308 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:15.252328 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:15.252332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:15.253535 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:15.253548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:15.254287 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:15.254294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:15.254519 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:15.255154 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:01:15.255156 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:15.255200 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:01:15.255299 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:15.255646 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:15.255657 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:15.255666 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 05:01:15.255817 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 05:01:15.256196 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":353,"threads_started":1}
I20260809 05:01:15.258736 30197 heartbeater.cc:347] Connected to a master server at 127.29.243.254:45775
I20260809 05:01:15.259052 30197 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:15.259956 30197 heartbeater.cc:511] Master 127.29.243.254:45775 requested a full tablet report, sending...
I20260809 05:01:15.262157 29886 ts_manager.cc:194] Registered new tserver with Master: 9e65fdb3d6224dc9b3a71974d61a54cb (127.29.243.193:34561)
I20260809 05:01:15.263720 29886 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.193:48273
W20260809 05:01:15.593102 30206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:15.593375 30206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:15.593511 30206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:15.600731 30206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:15.600832 30206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:15.600972 30206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.194
I20260809 05:01:15.610466 30206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:35651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.243.194
--webserver_port=0
--tserver_master_addrs=127.29.243.254:45775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.194
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:15.611877 30206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:15.613701 30206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:15.626556 30214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:15.626653 30213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:15.626626 30218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:15.627566 30206 server_base.cc:1034] running on GCE node
I20260809 05:01:15.628110 30206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:15.629029 30206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:15.630273 30206 hybrid_clock.cc:648] HybridClock initialized: now 1786251675630215 us; error 45 us; skew 500 ppm
I20260809 05:01:15.630630 30206 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:01:15.632853 30206 webserver.cc:460] Webserver started at http://127.29.243.194:34919/ using document root <none> and password file <none>
I20260809 05:01:15.633533 30206 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:15.633637 30206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:15.633946 30206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:15.636971 30206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/data/instance:
uuid: "496e39df773c4508b5c33b3913b2c88f"
format_stamp: "Formatted at 2026-08-09 05:01:15 on dist-test-slave-21b9"
server_key: "b1a1b5ef643f3aac8697f9aef2415f05"
server_key_iv: "590ce830052b34602540533ca0ef0291"
server_key_version: "encryptionkey@0"
I20260809 05:01:15.637727 30206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance:
uuid: "496e39df773c4508b5c33b3913b2c88f"
format_stamp: "Formatted at 2026-08-09 05:01:15 on dist-test-slave-21b9"
server_key: "b1a1b5ef643f3aac8697f9aef2415f05"
server_key_iv: "590ce830052b34602540533ca0ef0291"
server_key_version: "encryptionkey@0"
I20260809 05:01:15.642823 30206 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:01:15.646258 30226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:15.647603 30206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:15.647756 30206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/wal
uuid: "496e39df773c4508b5c33b3913b2c88f"
format_stamp: "Formatted at 2026-08-09 05:01:15 on dist-test-slave-21b9"
server_key: "b1a1b5ef643f3aac8697f9aef2415f05"
server_key_iv: "590ce830052b34602540533ca0ef0291"
server_key_version: "encryptionkey@0"
I20260809 05:01:15.647948 30206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:15.661269 30206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:15.662050 30206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:15.662331 30206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:15.663254 30206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:15.664842 30206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:15.664952 30206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:15.665061 30206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:15.665132 30206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:15.697142 30206 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.194:35053
I20260809 05:01:15.697190 30402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.194:35053 every 8 connection(s)
I20260809 05:01:15.698506 30206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/data/info.pb
I20260809 05:01:15.700011 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 30206
I20260809 05:01:15.700212 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/wal/instance
I20260809 05:01:15.700786 30671 external_mini_cluster.cc:1428] Setting key 9b8b9fc54e151086acbdd384d86b752f
I20260809 05:01:15.703881 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.243.195:0
--local_ip_for_outbound_sockets=127.29.243.195
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:45775
--builtin_ntp_servers=127.29.243.212:35651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:01:15.711534 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:15.701434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:54493 (local address 127.29.243.254:45775)
0809 05:01:15.701658 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:15.701664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:15.703254 (+ 1590us) server_negotiation.cc:407] Connection header received
0809 05:01:15.705180 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:15.705186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:15.705272 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:15.705359 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:15.706529 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:15.707241 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:15.708830 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:15.709243 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:15.709356 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:15.710700 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:15.710705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:15.710708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:15.710727 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:15.710760 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:15.710772 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:15.710848 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:15.711075 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 05:01:15.711184 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 05:01:15.712574 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:15.702019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:45775 (local address 127.29.243.194:54493)
0809 05:01:15.703117 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:15.703155 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:15.704900 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:15.705617 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:15.705626 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:15.705717 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:15.706359 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:15.706379 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:15.707425 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:15.707429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:15.708680 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:15.708693 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:15.709471 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:15.709480 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:15.709712 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:15.710380 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:01:15.710383 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:15.710424 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:01:15.710516 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:15.710863 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:15.710875 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:15.710883 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 05:01:15.711049 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 05:01:15.711478 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":338,"threads_started":1}
I20260809 05:01:15.714005 30403 heartbeater.cc:347] Connected to a master server at 127.29.243.254:45775
I20260809 05:01:15.714279 30403 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:15.715133 30403 heartbeater.cc:511] Master 127.29.243.254:45775 requested a full tablet report, sending...
I20260809 05:01:15.716594 29886 ts_manager.cc:194] Registered new tserver with Master: 496e39df773c4508b5c33b3913b2c88f (127.29.243.194:35053)
I20260809 05:01:15.717581 29886 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.194:54493
W20260809 05:01:16.052829 30407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:16.053124 30407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:16.053236 30407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:16.060715 30407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:16.060848 30407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:16.061004 30407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.243.195
I20260809 05:01:16.071213 30407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:35651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.243.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.243.195
--webserver_port=0
--tserver_master_addrs=127.29.243.254:45775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.243.195
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:16.072605 30407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:16.074484 30407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:16.087749 30415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:16.087793 30420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:16.087831 30416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:16.088773 30407 server_base.cc:1034] running on GCE node
I20260809 05:01:16.089336 30407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:16.090222 30407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:16.091420 30407 hybrid_clock.cc:648] HybridClock initialized: now 1786251676091366 us; error 46 us; skew 500 ppm
I20260809 05:01:16.091740 30407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:01:16.093902 30407 webserver.cc:460] Webserver started at http://127.29.243.195:42563/ using document root <none> and password file <none>
I20260809 05:01:16.094623 30407 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:16.094730 30407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:16.095041 30407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:16.097965 30407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/data/instance:
uuid: "72d211d4050646948d1cbbe9d5cfafaa"
format_stamp: "Formatted at 2026-08-09 05:01:16 on dist-test-slave-21b9"
server_key: "669a476ef59c7f899623ad0cfe1aa1d2"
server_key_iv: "bf6d5a7bd00bc3f26bdec227b905c4ca"
server_key_version: "encryptionkey@0"
I20260809 05:01:16.098729 30407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance:
uuid: "72d211d4050646948d1cbbe9d5cfafaa"
format_stamp: "Formatted at 2026-08-09 05:01:16 on dist-test-slave-21b9"
server_key: "669a476ef59c7f899623ad0cfe1aa1d2"
server_key_iv: "bf6d5a7bd00bc3f26bdec227b905c4ca"
server_key_version: "encryptionkey@0"
I20260809 05:01:16.103855 30407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 05:01:16.107848 30435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:16.109200 30407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:01:16.109354 30407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/wal
uuid: "72d211d4050646948d1cbbe9d5cfafaa"
format_stamp: "Formatted at 2026-08-09 05:01:16 on dist-test-slave-21b9"
server_key: "669a476ef59c7f899623ad0cfe1aa1d2"
server_key_iv: "bf6d5a7bd00bc3f26bdec227b905c4ca"
server_key_version: "encryptionkey@0"
I20260809 05:01:16.109594 30407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:16.150792 30407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:16.151757 30407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:16.152055 30407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:16.153070 30407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:01:16.154605 30407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:01:16.154706 30407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:16.154810 30407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:01:16.154886 30407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:16.185798 30407 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.195:34331
I20260809 05:01:16.185850 30615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.195:34331 every 8 connection(s)
I20260809 05:01:16.187135 30407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/data/info.pb
I20260809 05:01:16.188282 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 30407
I20260809 05:01:16.188475 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/wal/instance
I20260809 05:01:16.189082 30671 external_mini_cluster.cc:1428] Setting key 4cb06d44dfb655a3bc098726d4308bf8
I20260809 05:01:16.199756 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.189954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.195:37879 (local address 127.29.243.254:45775)
0809 05:01:16.190121 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:16.190125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:16.191723 (+ 1598us) server_negotiation.cc:407] Connection header received
0809 05:01:16.193576 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:16.193582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:16.193663 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:16.193746 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:16.194814 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.195601 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.197117 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.197534 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.197643 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.198918 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:16.198923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:16.198926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:16.198944 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:16.198976 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:16.198988 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:16.199064 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:16.199296 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 05:01:16.199399 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 05:01:16.200690 30621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.190626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:45775 (local address 127.29.243.195:37879)
0809 05:01:16.191591 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:16.191626 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:16.193316 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:16.194001 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:16.194009 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:16.194093 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:16.194654 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.194671 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.195756 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.195761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.196969 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.196982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.197752 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.197760 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.197969 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.198613 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:01:16.198615 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:16.198654 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:01:16.198736 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:16.199078 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:16.199090 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:16.199098 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 05:01:16.199245 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 05:01:16.199645 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":304,"threads_started":1}
I20260809 05:01:16.201959 30616 heartbeater.cc:347] Connected to a master server at 127.29.243.254:45775
I20260809 05:01:16.202231 30616 heartbeater.cc:464] Registering TS with master...
I20260809 05:01:16.203161 30616 heartbeater.cc:511] Master 127.29.243.254:45775 requested a full tablet report, sending...
I20260809 05:01:16.204641 29886 ts_manager.cc:194] Registered new tserver with Master: 72d211d4050646948d1cbbe9d5cfafaa (127.29.243.195:34331)
I20260809 05:01:16.205600 29886 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.243.195:37879
I20260809 05:01:16.216868 30671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:01:16.226660 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.221125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54368 (local address 127.29.243.254:45775)
0809 05:01:16.221315 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:16.221323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:16.221748 (+ 425us) server_negotiation.cc:407] Connection header received
0809 05:01:16.221920 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:16.221925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:16.222021 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:16.222117 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:16.222536 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.223210 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.224572 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.224989 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.225083 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.225982 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:16.225987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:16.225991 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:16.226007 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:16.226040 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:16.226051 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:16.226117 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:16.226284 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 05:01:16.226361 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 05:01:16.230379 29886 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:01:16.237362 29886 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:01:16.261011 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.249953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:34561 (local address 127.0.0.1:46076)
0809 05:01:16.251252 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:16.251318 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:16.251514 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:16.252733 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:16.252741 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:16.252774 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:16.253215 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.253228 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.255437 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.255443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.257780 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.257797 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.260054 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.260061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.260280 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.260331 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 05:01:16.260456 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 05:01:16.260649 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"thread_start_us":858,"threads_started":1}
I20260809 05:01:16.262130 30634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.250149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46076 (local address 127.29.243.193:34561)
0809 05:01:16.250792 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:16.250814 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:16.251431 (+ 617us) server_negotiation.cc:407] Connection header received
0809 05:01:16.252095 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:16.252104 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:16.252464 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:16.253408 (+ 944us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:16.253628 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.255243 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.258006 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.259487 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.259675 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.259921 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:16.261539 (+ 1618us) server_negotiation.cc:299] Negotiation successful
0809 05:01:16.261738 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":328,"threads_started":1}
I20260809 05:01:16.266700 30197 heartbeater.cc:503] Master 127.29.243.254:45775 was elected leader, sending a full tablet report...
I20260809 05:01:16.268428 30087 tablet_service.cc:1467] Processing CreateTablet for tablet 670202d3f9254e4c86dfe2e977a3b300 (DEFAULT_TABLE table=test-table [id=412625f960124fd9b7fbfd978097a528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:16.270133 30087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 670202d3f9254e4c86dfe2e977a3b300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:16.272390 30639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.251872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:34331 (local address 127.0.0.1:49342)
0809 05:01:16.264243 (+ 12371us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:16.264261 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:16.264387 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:16.265097 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:16.265102 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:16.265899 (+ 797us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:16.266116 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.266124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.267891 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.267896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.269789 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.269802 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.271690 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.271697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.271842 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.271870 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 05:01:16.271954 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 05:01:16.272082 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12249,"thread_start_us":12116,"threads_started":1}
I20260809 05:01:16.272742 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.252264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49342 (local address 127.29.243.195:34331)
0809 05:01:16.263827 (+ 11563us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:16.263836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:16.264506 (+ 670us) server_negotiation.cc:407] Connection header received
0809 05:01:16.264614 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:16.264623 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:16.264889 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:16.265079 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:16.266298 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.267711 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.269978 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.271123 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.271340 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.271568 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:16.272023 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 05:01:16.272192 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11375,"thread_start_us":11210,"threads_started":1}
I20260809 05:01:16.273824 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.250553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.194:35053 (local address 127.0.0.1:50888)
0809 05:01:16.261259 (+ 10706us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:16.261279 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:16.261372 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:16.267135 (+ 5763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:16.267140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:16.267165 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:16.267425 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.267437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.269595 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.269601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.271413 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.271429 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.273135 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.273141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.273314 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.273348 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 05:01:16.273429 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 05:01:16.273561 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10595,"thread_start_us":14846,"threads_started":1}
I20260809 05:01:16.274001 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.250304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50888 (local address 127.29.243.194:35053)
0809 05:01:16.265969 (+ 15665us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:16.265977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:16.266017 (+ 40us) server_negotiation.cc:407] Connection header received
0809 05:01:16.266121 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:16.266128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:16.266342 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:16.266547 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:16.267656 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.269233 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.271611 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.272955 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.273130 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.273326 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:16.273490 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 05:01:16.273649 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15509,"thread_start_us":15389,"threads_started":1}
I20260809 05:01:16.276636 30512 tablet_service.cc:1467] Processing CreateTablet for tablet 670202d3f9254e4c86dfe2e977a3b300 (DEFAULT_TABLE table=test-table [id=412625f960124fd9b7fbfd978097a528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:16.277319 30299 tablet_service.cc:1467] Processing CreateTablet for tablet 670202d3f9254e4c86dfe2e977a3b300 (DEFAULT_TABLE table=test-table [id=412625f960124fd9b7fbfd978097a528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:01:16.277824 30512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 670202d3f9254e4c86dfe2e977a3b300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:16.278342 30299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 670202d3f9254e4c86dfe2e977a3b300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:16.295045 30644 tablet_bootstrap.cc:492] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f: Bootstrap starting.
I20260809 05:01:16.295140 30645 tablet_bootstrap.cc:492] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa: Bootstrap starting.
I20260809 05:01:16.298763 30643 tablet_bootstrap.cc:492] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb: Bootstrap starting.
I20260809 05:01:16.299067 30644 tablet_bootstrap.cc:654] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:16.300424 30644 log.cc:824] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:16.300843 30645 tablet_bootstrap.cc:654] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:16.302065 30645 log.cc:824] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:16.303562 30643 tablet_bootstrap.cc:654] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:16.303881 30644 tablet_bootstrap.cc:492] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f: No bootstrap required, opened a new log
I20260809 05:01:16.304168 30644 ts_tablet_manager.cc:1397] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 05:01:16.304991 30645 tablet_bootstrap.cc:492] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa: No bootstrap required, opened a new log
I20260809 05:01:16.305308 30645 ts_tablet_manager.cc:1397] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 05:01:16.305305 30643 log.cc:824] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:16.308301 30643 tablet_bootstrap.cc:492] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb: No bootstrap required, opened a new log
I20260809 05:01:16.308570 30643 ts_tablet_manager.cc:1397] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 05:01:16.312623 30645 raft_consensus.cc:348] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34561 } } peers { permanent_uuid: "496e39df773c4508b5c33b3913b2c88f" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35053 } } peers { permanent_uuid: "72d211d4050646948d1cbbe9d5cfafaa" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 34331 } }
I20260809 05:01:16.312996 30645 raft_consensus.cc:374] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:16.313102 30645 raft_consensus.cc:729] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72d211d4050646948d1cbbe9d5cfafaa, State: Initialized, Role: FOLLOWER
I20260809 05:01:16.313735 30645 consensus_queue.cc:260] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34561 } } peers { permanent_uuid: "496e39df773c4508b5c33b3913b2c88f" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35053 } } peers { permanent_uuid: "72d211d4050646948d1cbbe9d5cfafaa" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 34331 } }
I20260809 05:01:16.314469 30644 raft_consensus.cc:348] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34561 } } peers { permanent_uuid: "496e39df773c4508b5c33b3913b2c88f" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35053 } } peers { permanent_uuid: "72d211d4050646948d1cbbe9d5cfafaa" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 34331 } }
I20260809 05:01:16.314900 30644 raft_consensus.cc:374] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:16.315001 30644 raft_consensus.cc:729] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496e39df773c4508b5c33b3913b2c88f, State: Initialized, Role: FOLLOWER
I20260809 05:01:16.315469 30645 ts_tablet_manager.cc:1428] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 05:01:16.315817 30616 heartbeater.cc:503] Master 127.29.243.254:45775 was elected leader, sending a full tablet report...
I20260809 05:01:16.315680 30644 consensus_queue.cc:260] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34561 } } peers { permanent_uuid: "496e39df773c4508b5c33b3913b2c88f" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35053 } } peers { permanent_uuid: "72d211d4050646948d1cbbe9d5cfafaa" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 34331 } }
I20260809 05:01:16.318965 30643 raft_consensus.cc:348] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34561 } } peers { permanent_uuid: "496e39df773c4508b5c33b3913b2c88f" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35053 } } peers { permanent_uuid: "72d211d4050646948d1cbbe9d5cfafaa" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 34331 } }
I20260809 05:01:16.319437 30643 raft_consensus.cc:374] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:16.319538 30643 raft_consensus.cc:729] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e65fdb3d6224dc9b3a71974d61a54cb, State: Initialized, Role: FOLLOWER
I20260809 05:01:16.320168 30643 consensus_queue.cc:260] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34561 } } peers { permanent_uuid: "496e39df773c4508b5c33b3913b2c88f" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35053 } } peers { permanent_uuid: "72d211d4050646948d1cbbe9d5cfafaa" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 34331 } }
I20260809 05:01:16.322101 30643 ts_tablet_manager.cc:1428] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260809 05:01:16.323056 30403 heartbeater.cc:503] Master 127.29.243.254:45775 was elected leader, sending a full tablet report...
I20260809 05:01:16.323589 30644 ts_tablet_manager.cc:1428] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f: Time spent starting tablet: real 0.019s user 0.011s sys 0.009s
I20260809 05:01:16.346809 30652 raft_consensus.cc:482] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:01:16.347114 30652 raft_consensus.cc:504] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34561 } } peers { permanent_uuid: "496e39df773c4508b5c33b3913b2c88f" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35053 } } peers { permanent_uuid: "72d211d4050646948d1cbbe9d5cfafaa" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 34331 } }
I20260809 05:01:16.348850 30652 leader_election.cc:288] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e65fdb3d6224dc9b3a71974d61a54cb (127.29.243.193:34561), 72d211d4050646948d1cbbe9d5cfafaa (127.29.243.195:34331)
I20260809 05:01:16.355327 30634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.349256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:36455 (local address 127.29.243.193:34561)
0809 05:01:16.349423 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:16.349428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:16.350031 (+ 603us) server_negotiation.cc:407] Connection header received
0809 05:01:16.350345 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:16.350351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:16.350439 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:16.350538 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:16.351109 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.351871 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.353674 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.354451 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.354598 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.354670 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:16.354937 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 05:01:16.355069 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 05:01:16.355399 30654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.349246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.193:34561 (local address 127.29.243.194:36455)
0809 05:01:16.349937 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:16.349957 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:16.350153 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:16.350584 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:16.350588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:16.350615 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:16.350970 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.350979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.351988 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.351993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.353486 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.353506 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.354634 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.354640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.354779 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.354803 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 05:01:16.354885 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 05:01:16.355007 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":18,"thread_start_us":373,"threads_started":1}
I20260809 05:01:16.356021 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.350385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.243.194:42427 (local address 127.29.243.195:34331)
0809 05:01:16.350582 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:16.350588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:16.350852 (+ 264us) server_negotiation.cc:407] Connection header received
0809 05:01:16.351018 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:16.351024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:16.351095 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:16.351175 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:01:16.351726 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.352563 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.354237 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.354993 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.355133 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.355200 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:16.355716 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 05:01:16.355841 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 05:01:16.356158 30656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.350113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.195:34331 (local address 127.29.243.194:42427)
0809 05:01:16.350754 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:16.350772 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:16.350891 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:16.351195 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:16.351199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:16.351221 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:01:16.351586 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.351596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.352696 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.352701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.354062 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:16.354074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.355174 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.355180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:16.355401 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.355432 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 05:01:16.355672 (+ 240us) client_negotiation.cc:240] Negotiation successful
0809 05:01:16.355794 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":18,"spinlock_wait_cycles":16256,"thread_start_us":466,"threads_started":1}
I20260809 05:01:16.356472 30126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "670202d3f9254e4c86dfe2e977a3b300" candidate_uuid: "496e39df773c4508b5c33b3913b2c88f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" is_pre_election: true
I20260809 05:01:16.357064 30126 raft_consensus.cc:2454] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496e39df773c4508b5c33b3913b2c88f in term 0.
I20260809 05:01:16.357138 30539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "670202d3f9254e4c86dfe2e977a3b300" candidate_uuid: "496e39df773c4508b5c33b3913b2c88f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72d211d4050646948d1cbbe9d5cfafaa" is_pre_election: true
I20260809 05:01:16.357627 30539 raft_consensus.cc:2454] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496e39df773c4508b5c33b3913b2c88f in term 0.
I20260809 05:01:16.357676 30229 leader_election.cc:302] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496e39df773c4508b5c33b3913b2c88f, 9e65fdb3d6224dc9b3a71974d61a54cb; no voters:
I20260809 05:01:16.358069 30652 raft_consensus.cc:2781] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:01:16.358228 30652 raft_consensus.cc:482] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:01:16.358338 30652 raft_consensus.cc:3037] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:16.360736 30652 raft_consensus.cc:504] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34561 } } peers { permanent_uuid: "496e39df773c4508b5c33b3913b2c88f" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35053 } } peers { permanent_uuid: "72d211d4050646948d1cbbe9d5cfafaa" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 34331 } }
I20260809 05:01:16.361294 30652 leader_election.cc:288] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [CANDIDATE]: Term 1 election: Requested vote from peers 9e65fdb3d6224dc9b3a71974d61a54cb (127.29.243.193:34561), 72d211d4050646948d1cbbe9d5cfafaa (127.29.243.195:34331)
I20260809 05:01:16.361791 30126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "670202d3f9254e4c86dfe2e977a3b300" candidate_uuid: "496e39df773c4508b5c33b3913b2c88f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb"
I20260809 05:01:16.361829 30539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "670202d3f9254e4c86dfe2e977a3b300" candidate_uuid: "496e39df773c4508b5c33b3913b2c88f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72d211d4050646948d1cbbe9d5cfafaa"
I20260809 05:01:16.362028 30539 raft_consensus.cc:3037] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:16.362028 30126 raft_consensus.cc:3037] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:16.365231 30539 raft_consensus.cc:2454] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496e39df773c4508b5c33b3913b2c88f in term 1.
I20260809 05:01:16.365231 30126 raft_consensus.cc:2454] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496e39df773c4508b5c33b3913b2c88f in term 1.
I20260809 05:01:16.365625 30229 leader_election.cc:302] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496e39df773c4508b5c33b3913b2c88f, 9e65fdb3d6224dc9b3a71974d61a54cb; no voters:
I20260809 05:01:16.365919 30652 raft_consensus.cc:2781] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:16.366397 30652 raft_consensus.cc:686] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 1 LEADER]: Becoming Leader. State: Replica: 496e39df773c4508b5c33b3913b2c88f, State: Running, Role: LEADER
I20260809 05:01:16.366954 30652 consensus_queue.cc:237] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34561 } } peers { permanent_uuid: "496e39df773c4508b5c33b3913b2c88f" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35053 } } peers { permanent_uuid: "72d211d4050646948d1cbbe9d5cfafaa" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 34331 } }
I20260809 05:01:16.371683 29885 catalog_manager.cc:5495] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f reported cstate change: term changed from 0 to 1, leader changed from <none> to 496e39df773c4508b5c33b3913b2c88f (127.29.243.194). New cstate: current_term: 1 leader_uuid: "496e39df773c4508b5c33b3913b2c88f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496e39df773c4508b5c33b3913b2c88f" member_type: VOTER last_known_addr { host: "127.29.243.194" port: 35053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72d211d4050646948d1cbbe9d5cfafaa" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 34331 } health_report { overall_health: UNKNOWN } } }
I20260809 05:01:16.400236 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.395011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50890 (local address 127.29.243.194:35053)
0809 05:01:16.395201 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:16.395206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:16.395398 (+ 192us) server_negotiation.cc:407] Connection header received
0809 05:01:16.395600 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:16.395604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:16.395700 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:16.395828 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:01:16.396139 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.396841 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.398192 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.398595 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.398686 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.399174 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:01:16.399758 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:01:16.399915 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:16.399955 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 05:01:16.400025 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:01:16.409173 30126 raft_consensus.cc:1261] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb [term 1 FOLLOWER]: Refusing update from remote peer 496e39df773c4508b5c33b3913b2c88f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:16.409175 30539 raft_consensus.cc:1261] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa [term 1 FOLLOWER]: Refusing update from remote peer 496e39df773c4508b5c33b3913b2c88f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:16.410137 30657 consensus_queue.cc:1035] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d211d4050646948d1cbbe9d5cfafaa" member_type: VOTER last_known_addr { host: "127.29.243.195" port: 34331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:01:16.410487 30652 consensus_queue.cc:1035] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e65fdb3d6224dc9b3a71974d61a54cb" member_type: VOTER last_known_addr { host: "127.29.243.193" port: 34561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:01:16.421838 30661 mvcc.cc:205] Tried to move back new op lower bound from 7316486866560651264 to 7316486866403557376. Current Snapshot: MvccSnapshot[applied={T|T < 7316486866560651264}]
I20260809 05:01:16.424691 30662 mvcc.cc:205] Tried to move back new op lower bound from 7316486866560651264 to 7316486866403557376. Current Snapshot: MvccSnapshot[applied={T|T < 7316486866560651264}]
I20260809 05:01:16.437252 29885 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:01:16.437556 29885 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:01:16.441509 29885 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b0ea6d1941844c2937c060a740d10ab: Sending DeleteTablet for 3 replicas of tablet 670202d3f9254e4c86dfe2e977a3b300
I20260809 05:01:16.442701 30299 tablet_service.cc:1514] Processing DeleteTablet for tablet 670202d3f9254e4c86dfe2e977a3b300 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:16 UTC) from {username='slave'} at 127.0.0.1:50888
I20260809 05:01:16.442698 30087 tablet_service.cc:1514] Processing DeleteTablet for tablet 670202d3f9254e4c86dfe2e977a3b300 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:16 UTC) from {username='slave'} at 127.0.0.1:46076
I20260809 05:01:16.442860 30512 tablet_service.cc:1514] Processing DeleteTablet for tablet 670202d3f9254e4c86dfe2e977a3b300 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:01:16 UTC) from {username='slave'} at 127.0.0.1:49342
I20260809 05:01:16.449314 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 29984
I20260809 05:01:16.455396 30683 tablet_replica.cc:330] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f: stopping tablet replica
I20260809 05:01:16.456053 30683 raft_consensus.cc:2229] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:01:16.457019 30683 raft_consensus.cc:2258] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:16.458033 30680 tablet_replica.cc:330] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb: stopping tablet replica
I20260809 05:01:16.458463 30680 raft_consensus.cc:2229] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:01:16.459020 30680 raft_consensus.cc:2258] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:16.459607 30683 ts_tablet_manager.cc:1905] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:16.461144 30680 ts_tablet_manager.cc:1905] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:16.467180 30683 ts_tablet_manager.cc:1918] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:16.467489 30683 log.cc:1196] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-1/wal/wals/670202d3f9254e4c86dfe2e977a3b300
I20260809 05:01:16.467798 30680 ts_tablet_manager.cc:1918] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:16.467954 30680 log.cc:1196] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-0/wal/wals/670202d3f9254e4c86dfe2e977a3b300
I20260809 05:01:16.468346 30683 ts_tablet_manager.cc:1939] T 670202d3f9254e4c86dfe2e977a3b300 P 496e39df773c4508b5c33b3913b2c88f: Deleting consensus metadata
I20260809 05:01:16.468433 30680 ts_tablet_manager.cc:1939] T 670202d3f9254e4c86dfe2e977a3b300 P 9e65fdb3d6224dc9b3a71974d61a54cb: Deleting consensus metadata
I20260809 05:01:16.470351 29869 catalog_manager.cc:4841] TS 9e65fdb3d6224dc9b3a71974d61a54cb (127.29.243.193:34561): tablet 670202d3f9254e4c86dfe2e977a3b300 (table test-table [id=412625f960124fd9b7fbfd978097a528]) successfully deleted
I20260809 05:01:16.470688 29866 catalog_manager.cc:4841] TS 496e39df773c4508b5c33b3913b2c88f (127.29.243.194:35053): tablet 670202d3f9254e4c86dfe2e977a3b300 (table test-table [id=412625f960124fd9b7fbfd978097a528]) successfully deleted
I20260809 05:01:16.471877 30684 tablet_replica.cc:330] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa: stopping tablet replica
I20260809 05:01:16.472316 30684 raft_consensus.cc:2229] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:01:16.472818 30684 raft_consensus.cc:2258] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:01:16.474647 30684 ts_tablet_manager.cc:1905] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:01:16.477802 30634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.451768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46092 (local address 127.29.243.193:34561)
0809 05:01:16.451970 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:16.451976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:16.469867 (+ 17891us) server_negotiation.cc:407] Connection header received
0809 05:01:16.470392 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:16.470398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:16.470516 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:16.470591 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:16.471264 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.472339 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.474292 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.474859 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.474999 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.476223 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:16.476234 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:16.476253 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:16.477006 (+ 753us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:16.477062 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:16.477087 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:16.477195 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:16.477351 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 05:01:16.477460 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 05:01:16.478541 30188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:16.481426 30684 ts_tablet_manager.cc:1918] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:01:16.481619 30684 log.cc:1196] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa: Deleting WAL directory at /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786251347713215-30671-0/minicluster-data/ts-2/wal/wals/670202d3f9254e4c86dfe2e977a3b300
I20260809 05:01:16.482149 30684 ts_tablet_manager.cc:1939] T 670202d3f9254e4c86dfe2e977a3b300 P 72d211d4050646948d1cbbe9d5cfafaa: Deleting consensus metadata
I20260809 05:01:16.484057 29869 catalog_manager.cc:4841] TS 72d211d4050646948d1cbbe9d5cfafaa (127.29.243.195:34331): tablet 670202d3f9254e4c86dfe2e977a3b300 (table test-table [id=412625f960124fd9b7fbfd978097a528]) successfully deleted
I20260809 05:01:16.567809 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 29984
I20260809 05:01:16.597918 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 30206
I20260809 05:01:16.605321 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.598680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37274 (local address 127.29.243.194:35053)
0809 05:01:16.598857 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:16.598863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:16.598946 (+ 83us) server_negotiation.cc:407] Connection header received
0809 05:01:16.599146 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:16.599152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:16.599249 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:16.599406 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:16.599817 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.600622 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.601884 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.602312 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.602422 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.603722 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:16.603732 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:16.603752 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:16.604448 (+ 696us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:16.604499 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:16.604520 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:16.604627 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:16.604762 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 05:01:16.604917 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 05:01:16.606056 30395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:16.717386 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 30206
I20260809 05:01:16.747316 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 30407
I20260809 05:01:16.754076 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:16.748098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39316 (local address 127.29.243.195:34331)
0809 05:01:16.748280 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:16.748286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:16.748307 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:01:16.748522 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:16.748528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:16.748616 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:16.748704 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:16.749142 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.749877 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.751197 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:16.751650 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:16.751762 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:16.752806 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:16.752816 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:16.752833 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:16.753407 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:16.753466 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:16.753493 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:16.753600 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:16.753737 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 05:01:16.753826 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 05:01:16.754714 30608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:16.842500 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 30407
I20260809 05:01:16.866374 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 29840
I20260809 05:01:16.867461 29936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:16.975309 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 29840
2026-08-09T05:01:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2841 ms)
[----------] 6 tests from EncryptionPolicyTest (22897 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:17 dist-test-slave-21b9 krb5kdc[30703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:17 dist-test-slave-21b9 krb5kdc[30703](info): set up 2 sockets
Aug 09 05:01:17 dist-test-slave-21b9 krb5kdc[30703](info): commencing operation
krb5kdc: starting...
W20260809 05:01:18.234850 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:18 dist-test-slave-21b9 krb5kdc[30703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:18.546838 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36705
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:18.886868 30728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:18.887188 30728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:18.887364 30728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:18.894537 30728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:18.894670 30728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:18.894754 30728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:18.894841 30728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:18.894905 30728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:18.904724 30728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36705
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:18.906031 30728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:18.907373 30728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:18.919237 30740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:18.919242 30738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:18.919289 30742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:18.920096 30728 server_base.cc:1034] running on GCE node
I20260809 05:01:18.920855 30728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:18.922231 30728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:18.923424 30728 hybrid_clock.cc:648] HybridClock initialized: now 1786251678923378 us; error 29 us; skew 500 ppm
I20260809 05:01:18.923775 30728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:18 dist-test-slave-21b9 krb5kdc[30703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:18.926839 30728 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:18.928368 30728 webserver.cc:460] Webserver started at http://127.0.0.1:42671/ using document root <none> and password file <none>
I20260809 05:01:18.929045 30728 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:18.929180 30728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:18.929474 30728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:18.932528 30728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "3626a0c7d4a345e8b472de9a46f5facd"
format_stamp: "Formatted at 2026-08-09 05:01:18 on dist-test-slave-21b9"
server_key: "68706c87333864e047f5620f243a397f"
server_key_iv: "176b9f976d20a239fcd77eddaa968847"
server_key_version: "encryptionkey@0"
I20260809 05:01:18.933290 30728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "3626a0c7d4a345e8b472de9a46f5facd"
format_stamp: "Formatted at 2026-08-09 05:01:18 on dist-test-slave-21b9"
server_key: "68706c87333864e047f5620f243a397f"
server_key_iv: "176b9f976d20a239fcd77eddaa968847"
server_key_version: "encryptionkey@0"
I20260809 05:01:18.938372 30728 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 05:01:18.941883 30750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:18.943197 30728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:01:18.943418 30728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "3626a0c7d4a345e8b472de9a46f5facd"
format_stamp: "Formatted at 2026-08-09 05:01:18 on dist-test-slave-21b9"
server_key: "68706c87333864e047f5620f243a397f"
server_key_iv: "176b9f976d20a239fcd77eddaa968847"
server_key_version: "encryptionkey@0"
I20260809 05:01:18.943594 30728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:18.967730 30728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:18.971850 30728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:18.972129 30728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:18.989876 30728 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36705
I20260809 05:01:18.989890 30825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36705 every 8 connection(s)
I20260809 05:01:18.991313 30728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:18.995301 30828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:18.997080 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 30728
I20260809 05:01:18.997324 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:18.998296 30671 external_mini_cluster.cc:1428] Setting key 425a46ad19124eca6ddf48250e101355
I20260809 05:01:19.008853 30828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd: Bootstrap starting.
I20260809 05:01:19.012984 30828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:19.014361 30828 log.cc:824] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd: Log is configured to *not* fsync() on all Append() calls
Aug 09 05:01:19 dist-test-slave-21b9 krb5kdc[30703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 05:01:19.017457 30828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd: No bootstrap required, opened a new log
I20260809 05:01:19.024060 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:19.000686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56849 (local address 127.0.0.1:36705)
0809 05:01:19.001707 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:19.001724 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:19.002416 (+ 692us) server_negotiation.cc:407] Connection header received
0809 05:01:19.004249 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:19.004274 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:19.004698 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:19.005022 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:19.006984 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:19.008456 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:19.010470 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:19.011222 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:19.011433 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:19.016585 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:19.016594 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:19.016610 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:19.016645 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:19.019775 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:19.020603 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:19.020610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:19.020617 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:19.020798 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:19.021249 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:19.021253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:19.021255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:19.021808 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:19.022146 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:19.022646 (+ 500us) server_negotiation.cc:299] Negotiation successful
0809 05:01:19.022965 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":389,"threads_started":1}
I20260809 05:01:19.026141 30828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3626a0c7d4a345e8b472de9a46f5facd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36705 } }
I20260809 05:01:19.026532 30828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:19.026680 30828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3626a0c7d4a345e8b472de9a46f5facd, State: Initialized, Role: FOLLOWER
I20260809 05:01:19.027360 30828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3626a0c7d4a345e8b472de9a46f5facd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36705 } }
I20260809 05:01:19.027684 30828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:19.027845 30828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:19.028045 30828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:19.028419 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:19.030822 30828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3626a0c7d4a345e8b472de9a46f5facd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36705 } }
I20260809 05:01:19.031388 30828 leader_election.cc:302] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3626a0c7d4a345e8b472de9a46f5facd; no voters:
I20260809 05:01:19.032018 30828 leader_election.cc:288] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:19.032171 30835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:19.033286 30835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [term 1 LEADER]: Becoming Leader. State: Replica: 3626a0c7d4a345e8b472de9a46f5facd, State: Running, Role: LEADER
I20260809 05:01:19.033995 30835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3626a0c7d4a345e8b472de9a46f5facd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36705 } }
I20260809 05:01:19.034247 30828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:19.039980 30838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3626a0c7d4a345e8b472de9a46f5facd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3626a0c7d4a345e8b472de9a46f5facd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36705 } } }
I20260809 05:01:19.040395 30838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:19.040910 30839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3626a0c7d4a345e8b472de9a46f5facd. Latest consensus state: current_term: 1 leader_uuid: "3626a0c7d4a345e8b472de9a46f5facd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3626a0c7d4a345e8b472de9a46f5facd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36705 } } }
I20260809 05:01:19.041210 30839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:19.043670 30855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:19.050418 30855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:19.065809 30855 catalog_manager.cc:1347] Generated new cluster ID: ccd95f8dab634500b94168912803e1e9
I20260809 05:01:19.067245 30855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:19.070420 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:19.044898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54843 (local address 127.0.0.1:36705)
0809 05:01:19.045086 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:19.045093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:19.051898 (+ 6805us) server_negotiation.cc:407] Connection header received
0809 05:01:19.052207 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:19.052214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:19.052302 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:19.052394 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:19.054551 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:19.055964 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:19.057848 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:19.058553 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:19.058671 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:19.062577 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:19.062587 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:19.062593 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:19.062645 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:19.066873 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:19.067857 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:19.067864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:19.067867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:19.068026 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:19.068614 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:19.068620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:19.068624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:19.069088 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:19.069393 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:19.069790 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 05:01:19.069983 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 05:01:19.092968 30855 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:19.094931 30855 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:19.114636 30855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3626a0c7d4a345e8b472de9a46f5facd: Generated new TSK 0
I20260809 05:01:19.115558 30855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:19.130184 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 30728
I20260809 05:01:19.132584 30819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:19.225224 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 30728
W20260809 05:01:19.250321 30701 connection.cc:535] client connection to 127.0.0.1:36705 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36705): Connection reset by peer (error 104)
2026-08-09T05:01:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2243 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:19 dist-test-slave-21b9 krb5kdc[30887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:19 dist-test-slave-21b9 krb5kdc[30887](info): set up 2 sockets
Aug 09 05:01:19 dist-test-slave-21b9 krb5kdc[30887](info): commencing operation
krb5kdc: starting...
W20260809 05:01:20.473969 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:20 dist-test-slave-21b9 krb5kdc[30887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:20.778687 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:21.124328 30908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:21.124631 30908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:21.124745 30908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:21.131881 30908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:21.131992 30908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:21.132046 30908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:21.132105 30908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:21.132149 30908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:21.142073 30908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32815
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:21.143401 30908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:21.144730 30908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:21.156025 30916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:21.156071 30917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:21.156113 30921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:21.157204 30908 server_base.cc:1034] running on GCE node
I20260809 05:01:21.157946 30908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:21.159412 30908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:21.160612 30908 hybrid_clock.cc:648] HybridClock initialized: now 1786251681160562 us; error 45 us; skew 500 ppm
I20260809 05:01:21.160948 30908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:21 dist-test-slave-21b9 krb5kdc[30887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:21.166811 30908 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:21.168370 30908 webserver.cc:460] Webserver started at http://127.0.0.1:40935/ using document root <none> and password file <none>
I20260809 05:01:21.169046 30908 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:21.169184 30908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:21.169485 30908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:21.172569 30908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "b8bef495a3a745cca1f576df351eb0b9"
format_stamp: "Formatted at 2026-08-09 05:01:21 on dist-test-slave-21b9"
server_key: "967cc4043fe7a326c7b00caa1895d8d7"
server_key_iv: "84358f8b99bb86def945ff717841d51d"
server_key_version: "encryptionkey@0"
I20260809 05:01:21.173319 30908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "b8bef495a3a745cca1f576df351eb0b9"
format_stamp: "Formatted at 2026-08-09 05:01:21 on dist-test-slave-21b9"
server_key: "967cc4043fe7a326c7b00caa1895d8d7"
server_key_iv: "84358f8b99bb86def945ff717841d51d"
server_key_version: "encryptionkey@0"
I20260809 05:01:21.178435 30908 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:01:21.181934 30935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:21.183372 30908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:21.183552 30908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "b8bef495a3a745cca1f576df351eb0b9"
format_stamp: "Formatted at 2026-08-09 05:01:21 on dist-test-slave-21b9"
server_key: "967cc4043fe7a326c7b00caa1895d8d7"
server_key_iv: "84358f8b99bb86def945ff717841d51d"
server_key_version: "encryptionkey@0"
I20260809 05:01:21.183790 30908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:21.196933 30908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:21.201010 30908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:21.201301 30908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:21.219803 30908 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32815
I20260809 05:01:21.219856 31014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32815 every 8 connection(s)
I20260809 05:01:21.221221 30908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:21.224956 31015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:21.229612 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 30908
I20260809 05:01:21.229791 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:21.230247 30671 external_mini_cluster.cc:1428] Setting key bc56ee2e15cd890ced9a268032bff2fd
I20260809 05:01:21.237063 31015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9: Bootstrap starting.
I20260809 05:01:21.240661 31015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:21.241981 31015 log.cc:824] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:21.245168 31015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9: No bootstrap required, opened a new log
I20260809 05:01:21.245174 31019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:21.232132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54839 (local address 127.0.0.1:32815)
0809 05:01:21.232999 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:21.233018 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:21.233070 (+ 52us) server_negotiation.cc:407] Connection header received
0809 05:01:21.235452 (+ 2382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:21.235480 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:21.236048 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:21.236466 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:21.237013 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:21.238963 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:21.240335 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:21.241366 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:21.241548 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:21.242644 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:21.242651 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:21.242665 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:21.243230 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:21.243335 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:21.243373 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:21.243526 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:21.243751 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 05:01:21.243915 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":383,"threads_started":1}
I20260809 05:01:21.249289 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:21.253799 31015 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bef495a3a745cca1f576df351eb0b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32815 } }
I20260809 05:01:21.254170 31015 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:21.254271 31015 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8bef495a3a745cca1f576df351eb0b9, State: Initialized, Role: FOLLOWER
I20260809 05:01:21.254853 31015 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bef495a3a745cca1f576df351eb0b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32815 } }
I20260809 05:01:21.255107 31015 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:21.255242 31015 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:21.255460 31015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:21.257584 31015 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bef495a3a745cca1f576df351eb0b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32815 } }
I20260809 05:01:21.258052 31015 leader_election.cc:302] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8bef495a3a745cca1f576df351eb0b9; no voters:
I20260809 05:01:21.258648 31015 leader_election.cc:288] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:21.258771 31032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:21.260223 31019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:21.253352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60757 (local address 127.0.0.1:32815)
0809 05:01:21.253499 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:21.253505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:21.253930 (+ 425us) server_negotiation.cc:407] Connection header received
0809 05:01:21.254112 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:21.254117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:21.254188 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:21.254272 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:21.254951 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:21.255920 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:21.257214 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:21.257653 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:21.257760 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:21.258897 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:21.258903 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:21.258908 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:21.258931 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:21.259040 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:21.259070 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:21.259203 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:21.259767 (+ 564us) server_negotiation.cc:299] Negotiation successful
0809 05:01:21.259817 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 05:01:21.260902 31015 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:21.260622 31032 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [term 1 LEADER]: Becoming Leader. State: Replica: b8bef495a3a745cca1f576df351eb0b9, State: Running, Role: LEADER
I20260809 05:01:21.261940 31032 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bef495a3a745cca1f576df351eb0b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32815 } }
I20260809 05:01:21.265057 31034 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8bef495a3a745cca1f576df351eb0b9. Latest consensus state: current_term: 1 leader_uuid: "b8bef495a3a745cca1f576df351eb0b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bef495a3a745cca1f576df351eb0b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32815 } } }
I20260809 05:01:21.265457 31034 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:21.265949 31033 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8bef495a3a745cca1f576df351eb0b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bef495a3a745cca1f576df351eb0b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32815 } } }
I20260809 05:01:21.266252 31033 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:21.267817 31055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:21.267948 31063 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 05:01:21.271886 31055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:21.279719 31055 catalog_manager.cc:1347] Generated new cluster ID: 578cf70e2d174386a9d6765a542f3a02
I20260809 05:01:21.279821 31055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:21.293699 31055 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:21.294860 31055 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:21.314414 31055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8bef495a3a745cca1f576df351eb0b9: Generated new TSK 0
I20260809 05:01:21.315536 31055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:21.325984 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 30908
W20260809 05:01:21.327576 31009 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54839
W20260809 05:01:21.328342 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 30908: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:21.328553 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 30908
W20260809 05:01:21.352073 30884 connection.cc:535] client connection to 127.0.0.1:32815 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32815): Connection reset by peer (error 104)
2026-08-09T05:01:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2102 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:21 dist-test-slave-21b9 krb5kdc[31073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:21 dist-test-slave-21b9 krb5kdc[31073](info): set up 2 sockets
Aug 09 05:01:21 dist-test-slave-21b9 krb5kdc[31073](info): commencing operation
krb5kdc: starting...
W20260809 05:01:22.564519 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:22 dist-test-slave-21b9 krb5kdc[31073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:22.876021 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34033
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:23.219290 31095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:23.219626 31095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:23.219769 31095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:23.226864 31095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:23.226985 31095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:23.227066 31095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:23.227144 31095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:23.227202 31095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:23.237000 31095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34033
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:23.238277 31095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:23.239609 31095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:23.250867 31117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:23.251065 31114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:23.251406 31111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:23.251989 31095 server_base.cc:1034] running on GCE node
I20260809 05:01:23.252815 31095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:23.254206 31095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:23.255391 31095 hybrid_clock.cc:648] HybridClock initialized: now 1786251683255355 us; error 63 us; skew 500 ppm
I20260809 05:01:23.255774 31095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:23 dist-test-slave-21b9 krb5kdc[31073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251683, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:23.258965 31095 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:23.260501 31095 webserver.cc:460] Webserver started at http://127.0.0.1:42423/ using document root <none> and password file <none>
I20260809 05:01:23.261191 31095 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:23.261317 31095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:23.261626 31095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:23.264708 31095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "fc0d187632994a77878a370b9b93c6b8"
format_stamp: "Formatted at 2026-08-09 05:01:23 on dist-test-slave-21b9"
server_key: "ce6153d670944b68536e13348b0caefa"
server_key_iv: "8f25a7fdf4afce860aa507de78e18052"
server_key_version: "encryptionkey@0"
I20260809 05:01:23.265506 31095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "fc0d187632994a77878a370b9b93c6b8"
format_stamp: "Formatted at 2026-08-09 05:01:23 on dist-test-slave-21b9"
server_key: "ce6153d670944b68536e13348b0caefa"
server_key_iv: "8f25a7fdf4afce860aa507de78e18052"
server_key_version: "encryptionkey@0"
I20260809 05:01:23.270754 31095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 05:01:23.274377 31128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:23.275820 31095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:23.275991 31095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "fc0d187632994a77878a370b9b93c6b8"
format_stamp: "Formatted at 2026-08-09 05:01:23 on dist-test-slave-21b9"
server_key: "ce6153d670944b68536e13348b0caefa"
server_key_iv: "8f25a7fdf4afce860aa507de78e18052"
server_key_version: "encryptionkey@0"
I20260809 05:01:23.276176 31095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:23.281929 31095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:23.285912 31095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:23.286183 31095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:23.304219 31095 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34033
I20260809 05:01:23.304268 31219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34033 every 8 connection(s)
I20260809 05:01:23.305629 31095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:23.307122 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 31095
I20260809 05:01:23.307322 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:23.307899 30671 external_mini_cluster.cc:1428] Setting key e44b79fc5abe61427944391ea12684d0
I20260809 05:01:23.310346 31220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:23.315615 31224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:23.309742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38281 (local address 127.0.0.1:34033)
0809 05:01:23.310553 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:23.310566 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:23.310612 (+ 46us) server_negotiation.cc:407] Connection header received
0809 05:01:23.312243 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:23.312266 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:23.312606 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:23.312953 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:23.313996 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:23.314006 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:23.314020 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:23.314034 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:23.314119 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:23.314142 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:23.314311 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:23.314424 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 05:01:23.314606 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":331,"threads_started":1}
I20260809 05:01:23.319182 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:23.322640 31220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8: Bootstrap starting.
I20260809 05:01:23.326404 31220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:23.327399 31224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:23.324712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37073 (local address 127.0.0.1:34033)
0809 05:01:23.324913 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:23.324922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:23.325324 (+ 402us) server_negotiation.cc:407] Connection header received
0809 05:01:23.325477 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:23.325482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:23.325543 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:23.325659 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:23.326659 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:23.326665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:23.326670 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:23.326693 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:23.326744 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:23.326766 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:23.326901 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:23.327039 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 05:01:23.327097 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:01:23.327751 31220 log.cc:824] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:23.330561 31220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8: No bootstrap required, opened a new log
I20260809 05:01:23.337951 31220 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0d187632994a77878a370b9b93c6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34033 } }
I20260809 05:01:23.338318 31220 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:23.338407 31220 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0d187632994a77878a370b9b93c6b8, State: Initialized, Role: FOLLOWER
I20260809 05:01:23.338917 31220 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0d187632994a77878a370b9b93c6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34033 } }
I20260809 05:01:23.339114 31220 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:23.339217 31220 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:23.339393 31220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:23.341732 31220 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0d187632994a77878a370b9b93c6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34033 } }
I20260809 05:01:23.342257 31220 leader_election.cc:302] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc0d187632994a77878a370b9b93c6b8; no voters:
I20260809 05:01:23.342836 31220 leader_election.cc:288] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:23.342973 31233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:23.343605 31233 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [term 1 LEADER]: Becoming Leader. State: Replica: fc0d187632994a77878a370b9b93c6b8, State: Running, Role: LEADER
I20260809 05:01:23.344169 31233 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0d187632994a77878a370b9b93c6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34033 } }
I20260809 05:01:23.345086 31220 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:23.346897 31235 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc0d187632994a77878a370b9b93c6b8. Latest consensus state: current_term: 1 leader_uuid: "fc0d187632994a77878a370b9b93c6b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0d187632994a77878a370b9b93c6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34033 } } }
I20260809 05:01:23.347059 31234 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc0d187632994a77878a370b9b93c6b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0d187632994a77878a370b9b93c6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34033 } } }
I20260809 05:01:23.347337 31235 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:23.347343 31234 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:23.350061 31240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:23.355391 31240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:23.364408 31240 catalog_manager.cc:1347] Generated new cluster ID: c3082a0d1a8641b4b0858bb9ded7ecda
I20260809 05:01:23.364552 31240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:23.406582 31240 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:23.407114 31240 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:23.421241 31240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc0d187632994a77878a370b9b93c6b8: Generated new TSK 0
I20260809 05:01:23.422164 31240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:23.427381 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31095
W20260809 05:01:23.429113 31213 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38281
W20260809 05:01:23.429850 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31095: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:23.429999 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 31095
2026-08-09T05:01:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2107 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:23 dist-test-slave-21b9 krb5kdc[31273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:23 dist-test-slave-21b9 krb5kdc[31273](info): set up 2 sockets
Aug 09 05:01:23 dist-test-slave-21b9 krb5kdc[31273](info): commencing operation
krb5kdc: starting...
W20260809 05:01:24.681102 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.004s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:24 dist-test-slave-21b9 krb5kdc[31273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:24.993921 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45997
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:25.334066 31291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:25.334368 31291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:25.334510 31291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:25.341691 31291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:25.341821 31291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:25.341903 31291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:25.341984 31291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:25.342047 31291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:25.351925 31291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:25.353238 31291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:25.354568 31291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:25.366060 31298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:25.366019 31301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:25.366093 31299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:25.366964 31291 server_base.cc:1034] running on GCE node
I20260809 05:01:25.367843 31291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:25.369285 31291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:25.370491 31291 hybrid_clock.cc:648] HybridClock initialized: now 1786251685370436 us; error 50 us; skew 500 ppm
I20260809 05:01:25.370831 31291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:25 dist-test-slave-21b9 krb5kdc[31273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251685, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:25.373970 31291 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:25.375406 31291 webserver.cc:460] Webserver started at http://127.0.0.1:46053/ using document root <none> and password file <none>
I20260809 05:01:25.376091 31291 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:25.376210 31291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:25.376513 31291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:25.379446 31291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "115925b2fb0e46d5ae7b31a01407f841"
format_stamp: "Formatted at 2026-08-09 05:01:25 on dist-test-slave-21b9"
server_key: "2f99a153ef590414e10c9f9338a15aa3"
server_key_iv: "4057121dcb44d71adbc0a5ef09e34f1a"
server_key_version: "encryptionkey@0"
I20260809 05:01:25.380205 31291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "115925b2fb0e46d5ae7b31a01407f841"
format_stamp: "Formatted at 2026-08-09 05:01:25 on dist-test-slave-21b9"
server_key: "2f99a153ef590414e10c9f9338a15aa3"
server_key_iv: "4057121dcb44d71adbc0a5ef09e34f1a"
server_key_version: "encryptionkey@0"
I20260809 05:01:25.385334 31291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:01:25.388760 31310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:25.390069 31291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:01:25.390239 31291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "115925b2fb0e46d5ae7b31a01407f841"
format_stamp: "Formatted at 2026-08-09 05:01:25 on dist-test-slave-21b9"
server_key: "2f99a153ef590414e10c9f9338a15aa3"
server_key_iv: "4057121dcb44d71adbc0a5ef09e34f1a"
server_key_version: "encryptionkey@0"
I20260809 05:01:25.390410 31291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:25.405205 31291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:25.409394 31291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:25.409672 31291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:25.427249 31291 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45997
I20260809 05:01:25.427295 31397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45997 every 8 connection(s)
I20260809 05:01:25.428684 31291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:25.432535 31399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:25.435395 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 31291
I20260809 05:01:25.435580 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:25.436106 30671 external_mini_cluster.cc:1428] Setting key 05b38b79c5732e3ecb26b5b9128b7089
I20260809 05:01:25.445820 31399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841: Bootstrap starting.
I20260809 05:01:25.449570 31399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:25.450600 31399 log.cc:824] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841: Log is configured to *not* fsync() on all Append() calls
Aug 09 05:01:25 dist-test-slave-21b9 krb5kdc[31273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 05:01:25.453756 31399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841: No bootstrap required, opened a new log
I20260809 05:01:25.461757 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:25.438136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34375 (local address 127.0.0.1:45997)
0809 05:01:25.439060 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:25.439077 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:25.439463 (+ 386us) server_negotiation.cc:407] Connection header received
0809 05:01:25.441598 (+ 2135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:25.441629 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:25.442240 (+ 611us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:25.442738 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:25.444231 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:25.446197 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:25.447580 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:25.448578 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:25.448764 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:25.453049 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:25.453062 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:25.453082 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:25.453128 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:25.457080 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:25.457807 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:25.457816 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:25.457829 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:25.458064 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:25.458417 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:25.458423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:25.458426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:25.459153 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:25.459480 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:25.459892 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 05:01:25.460365 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":392,"threads_started":1}
I20260809 05:01:25.463881 31399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115925b2fb0e46d5ae7b31a01407f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } }
I20260809 05:01:25.464244 31399 raft_consensus.cc:374] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:25.464349 31399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115925b2fb0e46d5ae7b31a01407f841, State: Initialized, Role: FOLLOWER
I20260809 05:01:25.464951 31399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115925b2fb0e46d5ae7b31a01407f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } }
I20260809 05:01:25.465189 31399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:25.465332 31399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:25.465515 31399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:25.465574 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:25.467835 31399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115925b2fb0e46d5ae7b31a01407f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } }
I20260809 05:01:25.468362 31399 leader_election.cc:302] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 115925b2fb0e46d5ae7b31a01407f841; no voters:
I20260809 05:01:25.468968 31399 leader_election.cc:288] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:25.469130 31412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:25.471410 31399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:25.471359 31412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [term 1 LEADER]: Becoming Leader. State: Replica: 115925b2fb0e46d5ae7b31a01407f841, State: Running, Role: LEADER
I20260809 05:01:25.472054 31412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115925b2fb0e46d5ae7b31a01407f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } }
I20260809 05:01:25.475066 31414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "115925b2fb0e46d5ae7b31a01407f841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115925b2fb0e46d5ae7b31a01407f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } } }
I20260809 05:01:25.475127 31415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 115925b2fb0e46d5ae7b31a01407f841. Latest consensus state: current_term: 1 leader_uuid: "115925b2fb0e46d5ae7b31a01407f841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "115925b2fb0e46d5ae7b31a01407f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } } }
I20260809 05:01:25.475518 31415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:25.475521 31414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:25.488435 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:25.470684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51217 (local address 127.0.0.1:45997)
0809 05:01:25.470851 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:25.470856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:25.470875 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:01:25.471092 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:25.471097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:25.471187 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:25.471311 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:25.473125 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:25.474025 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:25.475802 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:25.476449 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:25.476573 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:25.480058 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:25.480067 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:25.480073 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:25.480118 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:25.483915 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:25.484854 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:25.484860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:25.484864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:25.485038 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:25.485509 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:25.485516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:25.485519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:25.486102 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:25.486350 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:25.487738 (+ 1388us) server_negotiation.cc:299] Negotiation successful
0809 05:01:25.488018 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 05:01:25.490964 31422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:25.496290 31422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:25.505126 31422 catalog_manager.cc:1347] Generated new cluster ID: 6293f565163a492d802897d1981efac9
I20260809 05:01:25.505249 31422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:25.514189 31422 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:25.515576 31422 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:25.526260 31422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 115925b2fb0e46d5ae7b31a01407f841: Generated new TSK 0
I20260809 05:01:25.527065 31422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:25.534442 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31291
I20260809 05:01:25.535476 31390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:25.625087 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 31291
2026-08-09T05:01:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2189 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:25 dist-test-slave-21b9 krb5kdc[31444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:25 dist-test-slave-21b9 krb5kdc[31444](info): set up 2 sockets
Aug 09 05:01:25 dist-test-slave-21b9 krb5kdc[31444](info): commencing operation
krb5kdc: starting...
W20260809 05:01:26.877162 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:27 dist-test-slave-21b9 krb5kdc[31444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:27.199734 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:27.536195 31464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:27.536489 31464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:27.536610 31464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:27.543674 31464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:27.543779 31464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:27.543857 31464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:27.543936 31464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:27.543994 31464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:27.553624 31464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:27.554952 31464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:27.556289 31464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:27.567886 31478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:27.567939 31476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:27.567803 31475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:27.569024 31464 server_base.cc:1034] running on GCE node
I20260809 05:01:27.569803 31464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:27.571314 31464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:27.572557 31464 hybrid_clock.cc:648] HybridClock initialized: now 1786251687572508 us; error 47 us; skew 500 ppm
I20260809 05:01:27.572898 31464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:27 dist-test-slave-21b9 krb5kdc[31444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251687, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:27.578663 31464 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:27.580153 31464 webserver.cc:460] Webserver started at http://127.0.0.1:38239/ using document root <none> and password file <none>
I20260809 05:01:27.580812 31464 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:27.580938 31464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:27.581233 31464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:27.584273 31464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "0f8a5f8433264be5a0d9a84f49e892a5"
format_stamp: "Formatted at 2026-08-09 05:01:27 on dist-test-slave-21b9"
server_key: "1dd7e9dca4976a39abe9272d5eb888db"
server_key_iv: "e9be1253dd30cdab5b00f837af3e7942"
server_key_version: "encryptionkey@0"
I20260809 05:01:27.585016 31464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "0f8a5f8433264be5a0d9a84f49e892a5"
format_stamp: "Formatted at 2026-08-09 05:01:27 on dist-test-slave-21b9"
server_key: "1dd7e9dca4976a39abe9272d5eb888db"
server_key_iv: "e9be1253dd30cdab5b00f837af3e7942"
server_key_version: "encryptionkey@0"
I20260809 05:01:27.590040 31464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:01:27.593493 31488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:27.594764 31464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:01:27.594944 31464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "0f8a5f8433264be5a0d9a84f49e892a5"
format_stamp: "Formatted at 2026-08-09 05:01:27 on dist-test-slave-21b9"
server_key: "1dd7e9dca4976a39abe9272d5eb888db"
server_key_iv: "e9be1253dd30cdab5b00f837af3e7942"
server_key_version: "encryptionkey@0"
I20260809 05:01:27.595118 31464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:27.617123 31464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:27.621344 31464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:27.621619 31464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:27.640255 31464 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43135
I20260809 05:01:27.640280 31573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43135 every 8 connection(s)
I20260809 05:01:27.641588 31464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:27.645365 31574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:27.649746 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 31464
I20260809 05:01:27.649943 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:27.650506 30671 external_mini_cluster.cc:1428] Setting key 37fdc3f68ebd401381c30d077492a2f1
I20260809 05:01:27.656982 31574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5: Bootstrap starting.
I20260809 05:01:27.660665 31574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:27.661722 31574 log.cc:824] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:27.664446 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:27.652481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57591 (local address 127.0.0.1:43135)
0809 05:01:27.653289 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:27.653301 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:27.653347 (+ 46us) server_negotiation.cc:407] Connection header received
0809 05:01:27.655381 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:27.655407 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:27.655804 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:27.656125 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:27.656774 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:27.658130 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:27.660013 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:27.660694 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:27.660880 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:27.661972 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:27.661980 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:27.661998 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:27.662719 (+ 721us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:27.662821 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:27.662862 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:27.662984 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:27.663203 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 05:01:27.663474 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":375,"threads_started":1}
I20260809 05:01:27.664885 31574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5: No bootstrap required, opened a new log
I20260809 05:01:27.668136 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:27.674252 31574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a5f8433264be5a0d9a84f49e892a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43135 } }
I20260809 05:01:27.674705 31574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:27.674809 31574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f8a5f8433264be5a0d9a84f49e892a5, State: Initialized, Role: FOLLOWER
I20260809 05:01:27.675588 31574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a5f8433264be5a0d9a84f49e892a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43135 } }
I20260809 05:01:27.675874 31574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:27.676043 31574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:27.676414 31574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:27.679371 31574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a5f8433264be5a0d9a84f49e892a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43135 } }
I20260809 05:01:27.680033 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:27.673380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35021 (local address 127.0.0.1:43135)
0809 05:01:27.673570 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:27.673577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:27.674030 (+ 453us) server_negotiation.cc:407] Connection header received
0809 05:01:27.674176 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:27.674181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:27.674264 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:27.674373 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:27.674867 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:27.675905 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:27.677284 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:27.678033 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:27.678195 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:27.679178 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:27.679185 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:27.679190 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:27.679214 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:27.679265 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:27.679315 (+ 50us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:27.679433 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:27.679576 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 05:01:27.679671 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 05:01:27.680012 31574 leader_election.cc:302] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f8a5f8433264be5a0d9a84f49e892a5; no voters:
I20260809 05:01:27.680729 31574 leader_election.cc:288] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:27.680788 31595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:27.682090 31595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [term 1 LEADER]: Becoming Leader. State: Replica: 0f8a5f8433264be5a0d9a84f49e892a5, State: Running, Role: LEADER
I20260809 05:01:27.682678 31595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a5f8433264be5a0d9a84f49e892a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43135 } }
I20260809 05:01:27.683069 31574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:27.685282 31598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f8a5f8433264be5a0d9a84f49e892a5. Latest consensus state: current_term: 1 leader_uuid: "0f8a5f8433264be5a0d9a84f49e892a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a5f8433264be5a0d9a84f49e892a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43135 } } }
I20260809 05:01:27.685688 31598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:27.686206 31596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f8a5f8433264be5a0d9a84f49e892a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a5f8433264be5a0d9a84f49e892a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43135 } } }
I20260809 05:01:27.686538 31596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:27.692229 31603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:27.698406 31603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:27.707669 31603 catalog_manager.cc:1347] Generated new cluster ID: 3cbeaee33d944092bd03decc0458cbe9
I20260809 05:01:27.707782 31603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:27.734203 31603 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:27.735731 31603 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:27.750223 31603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f8a5f8433264be5a0d9a84f49e892a5: Generated new TSK 0
I20260809 05:01:27.751082 31603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:27.765611 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31464
W20260809 05:01:27.767181 31565 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57591
W20260809 05:01:27.768018 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31464: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:27.768185 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 31464
2026-08-09T05:01:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2144 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:27 dist-test-slave-21b9 krb5kdc[31626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:27 dist-test-slave-21b9 krb5kdc[31626](info): set up 2 sockets
Aug 09 05:01:27 dist-test-slave-21b9 krb5kdc[31626](info): commencing operation
krb5kdc: starting...
W20260809 05:01:29.012472 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:29 dist-test-slave-21b9 krb5kdc[31626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:29.332223 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37177
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:29.674494 31648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:29.674777 31648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:29.674901 31648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:29.682090 31648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:29.682201 31648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:29.682272 31648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:29.682353 31648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:29.682420 31648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:29.692236 31648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:29.693538 31648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:29.694857 31648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:29.712599 31653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:29.712527 31656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:29.712561 31658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:29.713374 31648 server_base.cc:1034] running on GCE node
I20260809 05:01:29.714134 31648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:29.715586 31648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:29.716777 31648 hybrid_clock.cc:648] HybridClock initialized: now 1786251689716735 us; error 37 us; skew 500 ppm
I20260809 05:01:29.717118 31648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:29 dist-test-slave-21b9 krb5kdc[31626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251689, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:29.720163 31648 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:29.721650 31648 webserver.cc:460] Webserver started at http://127.0.0.1:34193/ using document root <none> and password file <none>
I20260809 05:01:29.722301 31648 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:29.722414 31648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:29.722723 31648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:29.725729 31648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "a3927a31784e499da24f1f8761712ef2"
format_stamp: "Formatted at 2026-08-09 05:01:29 on dist-test-slave-21b9"
server_key: "514ad052ca1019a967bb98a673a29aa0"
server_key_iv: "3479f34dca8adfd0db2620064bca6349"
server_key_version: "encryptionkey@0"
I20260809 05:01:29.726464 31648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "a3927a31784e499da24f1f8761712ef2"
format_stamp: "Formatted at 2026-08-09 05:01:29 on dist-test-slave-21b9"
server_key: "514ad052ca1019a967bb98a673a29aa0"
server_key_iv: "3479f34dca8adfd0db2620064bca6349"
server_key_version: "encryptionkey@0"
I20260809 05:01:29.731741 31648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 05:01:29.735176 31674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:29.736476 31648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:29.736634 31648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "a3927a31784e499da24f1f8761712ef2"
format_stamp: "Formatted at 2026-08-09 05:01:29 on dist-test-slave-21b9"
server_key: "514ad052ca1019a967bb98a673a29aa0"
server_key_iv: "3479f34dca8adfd0db2620064bca6349"
server_key_version: "encryptionkey@0"
I20260809 05:01:29.736810 31648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:29.742484 31648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:29.746414 31648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:29.746685 31648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:29.763727 31648 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37177
I20260809 05:01:29.763767 31755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37177 every 8 connection(s)
I20260809 05:01:29.765062 31648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:29.768860 31760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:29.772859 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 31648
I20260809 05:01:29.773079 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:29.773607 30671 external_mini_cluster.cc:1428] Setting key 7b60fa78e03a33834d91b28c5988b08a
I20260809 05:01:29.781930 31760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2: Bootstrap starting.
I20260809 05:01:29.784931 31760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:29.786084 31760 log.cc:824] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:29.786890 31765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:29.775465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42517 (local address 127.0.0.1:37177)
0809 05:01:29.776276 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:29.776291 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:29.776344 (+ 53us) server_negotiation.cc:407] Connection header received
0809 05:01:29.778114 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:29.778144 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:29.778590 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:29.779048 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:29.785227 (+ 6179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:29.785234 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:29.785251 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:29.785269 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:29.785343 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:29.785366 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:29.785545 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:29.785714 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 05:01:29.785914 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":331,"threads_started":1}
I20260809 05:01:29.788893 31760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2: No bootstrap required, opened a new log
I20260809 05:01:29.790436 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:29.798029 31765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:29.795411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56847 (local address 127.0.0.1:37177)
0809 05:01:29.795586 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:29.795596 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:29.796020 (+ 424us) server_negotiation.cc:407] Connection header received
0809 05:01:29.796180 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:29.796184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:29.796247 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:29.796339 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:29.797423 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:29.797429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:29.797434 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:29.797453 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:29.797504 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:29.797524 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:29.797649 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:29.797692 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 05:01:29.797750 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 05:01:29.798655 31760 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3927a31784e499da24f1f8761712ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37177 } }
I20260809 05:01:29.799015 31760 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:29.799122 31760 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3927a31784e499da24f1f8761712ef2, State: Initialized, Role: FOLLOWER
I20260809 05:01:29.799754 31760 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3927a31784e499da24f1f8761712ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37177 } }
I20260809 05:01:29.800001 31760 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:29.800129 31760 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:29.800283 31760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:29.802512 31760 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3927a31784e499da24f1f8761712ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37177 } }
I20260809 05:01:29.802978 31760 leader_election.cc:302] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3927a31784e499da24f1f8761712ef2; no voters:
I20260809 05:01:29.803555 31760 leader_election.cc:288] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:29.803660 31777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:29.804287 31777 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [term 1 LEADER]: Becoming Leader. State: Replica: a3927a31784e499da24f1f8761712ef2, State: Running, Role: LEADER
I20260809 05:01:29.804890 31777 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3927a31784e499da24f1f8761712ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37177 } }
I20260809 05:01:29.805730 31760 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:29.807582 31780 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3927a31784e499da24f1f8761712ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3927a31784e499da24f1f8761712ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37177 } } }
I20260809 05:01:29.808295 31780 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:29.808151 31781 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3927a31784e499da24f1f8761712ef2. Latest consensus state: current_term: 1 leader_uuid: "a3927a31784e499da24f1f8761712ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3927a31784e499da24f1f8761712ef2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37177 } } }
I20260809 05:01:29.808625 31781 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:29.816654 31790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:29.822086 31790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:29.830766 31790 catalog_manager.cc:1347] Generated new cluster ID: fd93a90371da47e7be50dafba59e95a4
I20260809 05:01:29.830878 31790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:29.859248 31790 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:29.859694 31790 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:29.864910 31790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3927a31784e499da24f1f8761712ef2: Generated new TSK 0
I20260809 05:01:29.865684 31790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:29.872176 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31648
W20260809 05:01:29.873373 31750 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42517
W20260809 05:01:29.874030 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31648: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:29.874156 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 31648
2026-08-09T05:01:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2109 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:30 dist-test-slave-21b9 krb5kdc[31811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:30 dist-test-slave-21b9 krb5kdc[31811](info): set up 2 sockets
Aug 09 05:01:30 dist-test-slave-21b9 krb5kdc[31811](info): commencing operation
krb5kdc: starting...
W20260809 05:01:31.140908 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:31 dist-test-slave-21b9 krb5kdc[31811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:31.459558 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42489
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:31.800328 31832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:31.800631 31832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:31.800773 31832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:31.807842 31832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:31.807971 31832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:31.808049 31832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:31.808133 31832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:31.808192 31832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:31.818042 31832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:31.819453 31832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:31.820819 31832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:31.832388 31838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:31.832379 31847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:31.832393 31839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:31.833221 31832 server_base.cc:1034] running on GCE node
I20260809 05:01:31.834056 31832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:31.835490 31832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:31.836704 31832 hybrid_clock.cc:648] HybridClock initialized: now 1786251691836651 us; error 45 us; skew 500 ppm
I20260809 05:01:31.837055 31832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:31 dist-test-slave-21b9 krb5kdc[31811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251691, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:31.840215 31832 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:31.841692 31832 webserver.cc:460] Webserver started at http://127.0.0.1:45929/ using document root <none> and password file <none>
I20260809 05:01:31.842329 31832 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:31.842442 31832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:31.842689 31832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:31.845667 31832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "467120a3c9534fdaaa3a0d7318cb8e33"
format_stamp: "Formatted at 2026-08-09 05:01:31 on dist-test-slave-21b9"
server_key: "94a850d3c88870a1e5236f010d353d56"
server_key_iv: "2eaf1ccc9100691e445b2fcf30eb3cc0"
server_key_version: "encryptionkey@0"
I20260809 05:01:31.846390 31832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "467120a3c9534fdaaa3a0d7318cb8e33"
format_stamp: "Formatted at 2026-08-09 05:01:31 on dist-test-slave-21b9"
server_key: "94a850d3c88870a1e5236f010d353d56"
server_key_iv: "2eaf1ccc9100691e445b2fcf30eb3cc0"
server_key_version: "encryptionkey@0"
I20260809 05:01:31.851516 31832 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 05:01:31.855011 31856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:31.856369 31832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:31.856514 31832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "467120a3c9534fdaaa3a0d7318cb8e33"
format_stamp: "Formatted at 2026-08-09 05:01:31 on dist-test-slave-21b9"
server_key: "94a850d3c88870a1e5236f010d353d56"
server_key_iv: "2eaf1ccc9100691e445b2fcf30eb3cc0"
server_key_version: "encryptionkey@0"
I20260809 05:01:31.856678 31832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:31.869571 31832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:31.873311 31832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:31.873544 31832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:31.890880 31832 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42489
I20260809 05:01:31.890892 31934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42489 every 8 connection(s)
I20260809 05:01:31.892272 31832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:31.896039 31936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:31.900204 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 31832
I20260809 05:01:31.900393 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:31.900820 30671 external_mini_cluster.cc:1428] Setting key be827af9e2a25a8bcf09452b271f177c
I20260809 05:01:31.907011 31936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33: Bootstrap starting.
I20260809 05:01:31.910722 31936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:31.911871 31936 log.cc:824] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:31.915112 31936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33: No bootstrap required, opened a new log
Aug 09 05:01:31 dist-test-slave-21b9 krb5kdc[31811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 05:01:31.922818 31936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467120a3c9534fdaaa3a0d7318cb8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } }
I20260809 05:01:31.923228 31936 raft_consensus.cc:374] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:31.923393 31936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 467120a3c9534fdaaa3a0d7318cb8e33, State: Initialized, Role: FOLLOWER
I20260809 05:01:31.923976 31936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467120a3c9534fdaaa3a0d7318cb8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } }
I20260809 05:01:31.924204 31936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:31.924336 31936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:31.924499 31936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:31.924542 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:31.902656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:46553 (local address 127.0.0.1:42489)
0809 05:01:31.903424 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:31.903436 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:31.903478 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:01:31.905494 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:31.905518 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:31.905895 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:31.906191 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:31.908026 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:31.909458 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:31.911338 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:31.912058 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:31.912185 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:31.916832 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:31.916839 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:31.916854 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:31.916901 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:31.920063 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:31.920862 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:31.920870 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:31.920887 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:31.921068 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:31.921543 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:31.921547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:31.921549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:31.922117 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:31.922408 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:31.922974 (+ 566us) server_negotiation.cc:299] Negotiation successful
0809 05:01:31.923348 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":301,"threads_started":1}
I20260809 05:01:31.927356 31936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467120a3c9534fdaaa3a0d7318cb8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } }
I20260809 05:01:31.927876 31936 leader_election.cc:302] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 467120a3c9534fdaaa3a0d7318cb8e33; no voters:
I20260809 05:01:31.928510 31936 leader_election.cc:288] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:31.928647 31944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:31.928700 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:31.930536 31936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:31.931596 31944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [term 1 LEADER]: Becoming Leader. State: Replica: 467120a3c9534fdaaa3a0d7318cb8e33, State: Running, Role: LEADER
I20260809 05:01:31.932204 31944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467120a3c9534fdaaa3a0d7318cb8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } }
I20260809 05:01:31.934814 31947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 467120a3c9534fdaaa3a0d7318cb8e33. Latest consensus state: current_term: 1 leader_uuid: "467120a3c9534fdaaa3a0d7318cb8e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467120a3c9534fdaaa3a0d7318cb8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } } }
I20260809 05:01:31.935176 31947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:31.935176 31945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "467120a3c9534fdaaa3a0d7318cb8e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467120a3c9534fdaaa3a0d7318cb8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } } }
I20260809 05:01:31.935516 31945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:31.948263 31960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:31.953493 31960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:31.963752 31960 catalog_manager.cc:1347] Generated new cluster ID: 647130cf17b7474688b5b76400e002f7
I20260809 05:01:31.963871 31960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:31.977020 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:31.958200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:49595 (local address 127.0.0.1:42489)
0809 05:01:31.958360 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:31.958366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:31.958507 (+ 141us) server_negotiation.cc:407] Connection header received
0809 05:01:31.958804 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:31.958810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:31.958902 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:31.958976 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:31.961404 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:31.962686 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:31.964702 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:31.965429 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:31.965538 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:31.969232 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:31.969240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:31.969246 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:31.969289 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:31.973065 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:31.974114 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:31.974120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:31.974123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:31.974289 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:31.974892 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:31.974898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:31.974902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:31.975355 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:31.975646 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:31.976374 (+ 728us) server_negotiation.cc:299] Negotiation successful
0809 05:01:31.976590 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 05:01:31.985844 31960 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:31.987087 31960 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:31.993691 31960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 467120a3c9534fdaaa3a0d7318cb8e33: Generated new TSK 0
I20260809 05:01:31.994580 31960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:32.003252 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 31832
I20260809 05:01:32.004397 31929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:32.094287 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 31832
W20260809 05:01:32.119374 31809 connection.cc:535] client connection to 127.0.0.1:42489 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42489): Connection reset by peer (error 104)
2026-08-09T05:01:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2215 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:32 dist-test-slave-21b9 krb5kdc[31988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:32 dist-test-slave-21b9 krb5kdc[31988](info): set up 2 sockets
Aug 09 05:01:32 dist-test-slave-21b9 krb5kdc[31988](info): commencing operation
krb5kdc: starting...
W20260809 05:01:33.338462 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.005s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:33 dist-test-slave-21b9 krb5kdc[31988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:33.666887 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:34.005756 32006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:34.006052 32006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:34.006156 32006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:34.015771 32006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:34.015923 32006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:34.016007 32006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:34.016095 32006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:34.016158 32006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:34.026327 32006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:34.027767 32006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:34.029176 32006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:34.040987 32014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:34.040946 32024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:34.040913 32016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:34.041887 32006 server_base.cc:1034] running on GCE node
I20260809 05:01:34.042697 32006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:34.044222 32006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:34.045419 32006 hybrid_clock.cc:648] HybridClock initialized: now 1786251694045371 us; error 42 us; skew 500 ppm
I20260809 05:01:34.045794 32006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:34 dist-test-slave-21b9 krb5kdc[31988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:34.051677 32006 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:34.053248 32006 webserver.cc:460] Webserver started at http://127.0.0.1:44039/ using document root <none> and password file <none>
I20260809 05:01:34.053956 32006 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:34.054080 32006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:34.054389 32006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:34.057482 32006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "a28f429096a0400fa7d32debf8785394"
format_stamp: "Formatted at 2026-08-09 05:01:34 on dist-test-slave-21b9"
server_key: "e7f36a6fd31d0e7ae5835b9973b8f924"
server_key_iv: "c7b0410baf8bb42f92d18af3803bcce2"
server_key_version: "encryptionkey@0"
I20260809 05:01:34.058251 32006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "a28f429096a0400fa7d32debf8785394"
format_stamp: "Formatted at 2026-08-09 05:01:34 on dist-test-slave-21b9"
server_key: "e7f36a6fd31d0e7ae5835b9973b8f924"
server_key_iv: "c7b0410baf8bb42f92d18af3803bcce2"
server_key_version: "encryptionkey@0"
I20260809 05:01:34.063575 32006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:01:34.067116 32032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:34.068490 32006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:34.068679 32006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "a28f429096a0400fa7d32debf8785394"
format_stamp: "Formatted at 2026-08-09 05:01:34 on dist-test-slave-21b9"
server_key: "e7f36a6fd31d0e7ae5835b9973b8f924"
server_key_iv: "c7b0410baf8bb42f92d18af3803bcce2"
server_key_version: "encryptionkey@0"
I20260809 05:01:34.068867 32006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:34.091620 32006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:34.095937 32006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:34.096238 32006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:34.113830 32006 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44835
I20260809 05:01:34.113853 32109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44835 every 8 connection(s)
I20260809 05:01:34.115195 32006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:34.118299 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 32006
I20260809 05:01:34.118520 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:34.118975 30671 external_mini_cluster.cc:1428] Setting key cdd94045f9372450cfa971b35992d30e
I20260809 05:01:34.119199 32110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:34.132615 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:34.120848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:33829 (local address 127.0.0.1:44835)
0809 05:01:34.121612 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:34.121624 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:34.121669 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:01:34.123749 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:34.123774 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:34.124191 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:34.124518 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:34.125154 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:34.126478 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:34.128277 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:34.128972 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:34.129116 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:34.130362 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:34.130372 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:34.130386 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:34.130939 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:34.131012 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:34.131036 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:34.131150 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:34.131450 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 05:01:34.131645 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":332,"threads_started":1}
I20260809 05:01:34.132640 32110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394: Bootstrap starting.
I20260809 05:01:34.136413 32110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:34.136605 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:34.137428 32110 log.cc:824] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:34.140509 32110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394: No bootstrap required, opened a new log
I20260809 05:01:34.148039 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:34.141012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:50705 (local address 127.0.0.1:44835)
0809 05:01:34.141188 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:34.141194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:34.141679 (+ 485us) server_negotiation.cc:407] Connection header received
0809 05:01:34.141874 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:34.141877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:34.141940 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:34.142015 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:34.142657 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:34.143474 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:34.145279 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:34.145868 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:34.145988 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:34.147235 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:34.147243 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:34.147246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:34.147263 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:34.147361 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:34.147384 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:34.147488 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:34.147657 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 05:01:34.147703 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 05:01:34.149821 32110 raft_consensus.cc:348] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28f429096a0400fa7d32debf8785394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260809 05:01:34.150188 32110 raft_consensus.cc:374] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:34.150295 32110 raft_consensus.cc:729] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a28f429096a0400fa7d32debf8785394, State: Initialized, Role: FOLLOWER
I20260809 05:01:34.150861 32110 consensus_queue.cc:260] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28f429096a0400fa7d32debf8785394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260809 05:01:34.151090 32110 raft_consensus.cc:388] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:34.151221 32110 raft_consensus.cc:482] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:34.151422 32110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:34.153556 32110 raft_consensus.cc:504] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28f429096a0400fa7d32debf8785394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260809 05:01:34.154043 32110 leader_election.cc:302] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a28f429096a0400fa7d32debf8785394; no voters:
I20260809 05:01:34.154583 32110 leader_election.cc:288] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:34.154711 32131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:34.155797 32131 raft_consensus.cc:686] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [term 1 LEADER]: Becoming Leader. State: Replica: a28f429096a0400fa7d32debf8785394, State: Running, Role: LEADER
I20260809 05:01:34.156334 32131 consensus_queue.cc:237] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28f429096a0400fa7d32debf8785394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260809 05:01:34.156747 32110 sys_catalog.cc:564] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:34.158927 32132 sys_catalog.cc:455] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a28f429096a0400fa7d32debf8785394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28f429096a0400fa7d32debf8785394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } } }
I20260809 05:01:34.159300 32132 sys_catalog.cc:458] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:34.160786 32142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:34.160642 32133 sys_catalog.cc:455] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a28f429096a0400fa7d32debf8785394. Latest consensus state: current_term: 1 leader_uuid: "a28f429096a0400fa7d32debf8785394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a28f429096a0400fa7d32debf8785394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } } }
I20260809 05:01:34.160929 32133 sys_catalog.cc:458] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:34.166543 32142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:34.181639 32142 catalog_manager.cc:1347] Generated new cluster ID: b540479ed29e4be1b63f9c903182b00b
I20260809 05:01:34.181761 32142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:34.191982 32142 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:34.193241 32142 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:34.199908 32142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a28f429096a0400fa7d32debf8785394: Generated new TSK 0
I20260809 05:01:34.200649 32142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:34.211768 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32006
W20260809 05:01:34.213085 32105 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.177:33829
W20260809 05:01:34.213842 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32006: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:34.213984 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 32006
W20260809 05:01:34.243571 31983 connection.cc:535] client connection to 127.0.0.1:44835 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44835): Connection reset by peer (error 104)
2026-08-09T05:01:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2124 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:34 dist-test-slave-21b9 krb5kdc[32164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:34 dist-test-slave-21b9 krb5kdc[32164](info): set up 2 sockets
Aug 09 05:01:34 dist-test-slave-21b9 krb5kdc[32164](info): commencing operation
krb5kdc: starting...
W20260809 05:01:35.461418 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:35 dist-test-slave-21b9 krb5kdc[32164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:35.784183 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34661
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:36.128491 32181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:36.128791 32181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:36.128917 32181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:36.136080 32181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:36.136194 32181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:36.136268 32181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:36.136353 32181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:36.136411 32181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:36.146802 32181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:36.148164 32181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:36.149533 32181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:36.161046 32188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:36.161196 32189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:36.161506 32192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:36.162205 32181 server_base.cc:1034] running on GCE node
I20260809 05:01:36.162976 32181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:36.164459 32181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:36.165678 32181 hybrid_clock.cc:648] HybridClock initialized: now 1786251696165629 us; error 42 us; skew 500 ppm
I20260809 05:01:36.166025 32181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:36 dist-test-slave-21b9 krb5kdc[32164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251696, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:36.169108 32181 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:36.170527 32181 webserver.cc:460] Webserver started at http://127.0.0.1:33453/ using document root <none> and password file <none>
I20260809 05:01:36.171195 32181 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:36.171335 32181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:36.171643 32181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:36.174576 32181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "0d3e355d242d4460abc1de1b6028367f"
format_stamp: "Formatted at 2026-08-09 05:01:36 on dist-test-slave-21b9"
server_key: "360eed82384999eeb023e70aa15a35db"
server_key_iv: "5016fa483c7693e0fa4de6ec3d317398"
server_key_version: "encryptionkey@0"
I20260809 05:01:36.175364 32181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "0d3e355d242d4460abc1de1b6028367f"
format_stamp: "Formatted at 2026-08-09 05:01:36 on dist-test-slave-21b9"
server_key: "360eed82384999eeb023e70aa15a35db"
server_key_iv: "5016fa483c7693e0fa4de6ec3d317398"
server_key_version: "encryptionkey@0"
I20260809 05:01:36.180668 32181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 05:01:36.184140 32204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:36.185436 32181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:01:36.185598 32181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "0d3e355d242d4460abc1de1b6028367f"
format_stamp: "Formatted at 2026-08-09 05:01:36 on dist-test-slave-21b9"
server_key: "360eed82384999eeb023e70aa15a35db"
server_key_iv: "5016fa483c7693e0fa4de6ec3d317398"
server_key_version: "encryptionkey@0"
I20260809 05:01:36.185777 32181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:36.191562 32181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:36.195537 32181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:36.195784 32181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:36.212633 32181 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34661
I20260809 05:01:36.212682 32287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34661 every 8 connection(s)
I20260809 05:01:36.213980 32181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:36.214759 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 32181
I20260809 05:01:36.214967 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:36.215441 30671 external_mini_cluster.cc:1428] Setting key 1c24c7a81263b3c49a09cd208b701ff1
I20260809 05:01:36.218608 32288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:36.224334 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:36.217321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:42269 (local address 127.0.0.1:34661)
0809 05:01:36.218149 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:36.218165 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:36.218219 (+ 54us) server_negotiation.cc:407] Connection header received
0809 05:01:36.220441 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:36.220468 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:36.220904 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:36.221344 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:36.222341 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:36.222350 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:36.222367 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:36.222384 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:36.222487 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:36.222519 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:36.222722 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:36.222864 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 05:01:36.223079 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":300,"threads_started":1}
I20260809 05:01:36.227998 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:36.232468 32288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f: Bootstrap starting.
I20260809 05:01:36.235504 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:36.233281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:54471 (local address 127.0.0.1:34661)
0809 05:01:36.233452 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:36.233457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:36.233714 (+ 257us) server_negotiation.cc:407] Connection header received
0809 05:01:36.233840 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:36.233844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:36.233913 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:36.234032 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:36.234861 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:36.234866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:36.234870 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:36.234895 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:36.234941 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:36.234961 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:36.235076 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:36.235119 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 05:01:36.235177 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 05:01:36.236269 32288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:36.237509 32288 log.cc:824] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:36.240137 32288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f: No bootstrap required, opened a new log
I20260809 05:01:36.247670 32288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3e355d242d4460abc1de1b6028367f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } }
I20260809 05:01:36.248119 32288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:36.248236 32288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d3e355d242d4460abc1de1b6028367f, State: Initialized, Role: FOLLOWER
I20260809 05:01:36.248860 32288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3e355d242d4460abc1de1b6028367f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } }
I20260809 05:01:36.249087 32288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:36.249187 32288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:36.249317 32288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:36.251394 32288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3e355d242d4460abc1de1b6028367f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } }
I20260809 05:01:36.251858 32288 leader_election.cc:302] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d3e355d242d4460abc1de1b6028367f; no voters:
I20260809 05:01:36.252388 32288 leader_election.cc:288] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:36.252509 32305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:36.253118 32305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [term 1 LEADER]: Becoming Leader. State: Replica: 0d3e355d242d4460abc1de1b6028367f, State: Running, Role: LEADER
I20260809 05:01:36.253634 32305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3e355d242d4460abc1de1b6028367f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } }
I20260809 05:01:36.254307 32288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:36.256366 32306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d3e355d242d4460abc1de1b6028367f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3e355d242d4460abc1de1b6028367f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } } }
I20260809 05:01:36.256775 32306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:36.257298 32307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d3e355d242d4460abc1de1b6028367f. Latest consensus state: current_term: 1 leader_uuid: "0d3e355d242d4460abc1de1b6028367f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3e355d242d4460abc1de1b6028367f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } } }
I20260809 05:01:36.257589 32307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:36.259042 32314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:36.263763 32314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:36.272068 32314 catalog_manager.cc:1347] Generated new cluster ID: b8f3f1f2de1a4b6cbea7cc8df62ae5c0
I20260809 05:01:36.272200 32314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:36.294503 32314 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:36.294920 32314 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:36.302297 32314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d3e355d242d4460abc1de1b6028367f: Generated new TSK 0
I20260809 05:01:36.303068 32314 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:36.311308 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32181
W20260809 05:01:36.312394 32281 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.177:42269
W20260809 05:01:36.313041 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32181: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:36.313199 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 32181
2026-08-09T05:01:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2114 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:36 dist-test-slave-21b9 krb5kdc[32338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:36 dist-test-slave-21b9 krb5kdc[32338](info): set up 2 sockets
Aug 09 05:01:36 dist-test-slave-21b9 krb5kdc[32338](info): commencing operation
krb5kdc: starting...
W20260809 05:01:37.568296 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:37 dist-test-slave-21b9 krb5kdc[32338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:37.884047 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46465
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:38.229589 32360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:38.229872 32360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:38.229995 32360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:38.237142 32360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:38.237247 32360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:38.237308 32360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:38.237396 32360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:38.237455 32360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:38.247154 32360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46465
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:38.248422 32360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:38.249760 32360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:38.261312 32370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:38.261253 32371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:38.261291 32374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:38.262128 32360 server_base.cc:1034] running on GCE node
I20260809 05:01:38.262818 32360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:38.264204 32360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:38.265439 32360 hybrid_clock.cc:648] HybridClock initialized: now 1786251698265390 us; error 44 us; skew 500 ppm
I20260809 05:01:38.265787 32360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:38 dist-test-slave-21b9 krb5kdc[32338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251698, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:38.268942 32360 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:38.270418 32360 webserver.cc:460] Webserver started at http://127.0.0.1:40903/ using document root <none> and password file <none>
I20260809 05:01:38.271085 32360 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:38.271214 32360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:38.271538 32360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:38.274529 32360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "6142819da9d946988989c222bba8bebd"
format_stamp: "Formatted at 2026-08-09 05:01:38 on dist-test-slave-21b9"
server_key: "d8a2918991d6e3a71067c48834f519f0"
server_key_iv: "8a07b031e577d9e81947b826cc75109f"
server_key_version: "encryptionkey@0"
I20260809 05:01:38.275297 32360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "6142819da9d946988989c222bba8bebd"
format_stamp: "Formatted at 2026-08-09 05:01:38 on dist-test-slave-21b9"
server_key: "d8a2918991d6e3a71067c48834f519f0"
server_key_iv: "8a07b031e577d9e81947b826cc75109f"
server_key_version: "encryptionkey@0"
I20260809 05:01:38.280453 32360 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 05:01:38.284090 32383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:38.285492 32360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:01:38.285645 32360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "6142819da9d946988989c222bba8bebd"
format_stamp: "Formatted at 2026-08-09 05:01:38 on dist-test-slave-21b9"
server_key: "d8a2918991d6e3a71067c48834f519f0"
server_key_iv: "8a07b031e577d9e81947b826cc75109f"
server_key_version: "encryptionkey@0"
I20260809 05:01:38.285811 32360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:38.309360 32360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:38.313665 32360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:38.313912 32360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:38.331761 32360 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46465
I20260809 05:01:38.331785 32470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46465 every 8 connection(s)
I20260809 05:01:38.333120 32360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:38.335594 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 32360
I20260809 05:01:38.335765 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:38.336215 30671 external_mini_cluster.cc:1428] Setting key f288bba3bbfcc98d3a4deea21edf33da
I20260809 05:01:38.337146 32472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:01:38 dist-test-slave-21b9 krb5kdc[32338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 05:01:38.350798 32472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd: Bootstrap starting.
I20260809 05:01:38.354852 32472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:38.356127 32472 log.cc:824] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:38.359548 32472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd: No bootstrap required, opened a new log
I20260809 05:01:38.359952 32478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:38.338053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:40203 (local address 127.0.0.1:46465)
0809 05:01:38.338837 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:38.338851 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:38.338904 (+ 53us) server_negotiation.cc:407] Connection header received
0809 05:01:38.340743 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:38.340772 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:38.341307 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:38.341754 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:38.343065 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:38.345102 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:38.346402 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:38.347425 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:38.347615 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:38.351627 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:38.351639 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:38.351658 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:38.351701 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:38.355534 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:38.356300 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:38.356309 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:38.356316 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:38.356525 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:38.357057 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:38.357063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:38.357065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:38.357676 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:38.357971 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:38.358424 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 05:01:38.358745 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":311,"threads_started":1}
I20260809 05:01:38.364032 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:38.367726 32472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6142819da9d946988989c222bba8bebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46465 } }
I20260809 05:01:38.368140 32472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:38.368259 32472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6142819da9d946988989c222bba8bebd, State: Initialized, Role: FOLLOWER
I20260809 05:01:38.368991 32472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6142819da9d946988989c222bba8bebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46465 } }
I20260809 05:01:38.369287 32472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:38.369447 32472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:38.369628 32472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:38.372416 32472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6142819da9d946988989c222bba8bebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46465 } }
I20260809 05:01:38.372987 32472 leader_election.cc:302] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6142819da9d946988989c222bba8bebd; no voters:
I20260809 05:01:38.373659 32472 leader_election.cc:288] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:38.373759 32487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:38.375967 32472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:38.378225 32487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [term 1 LEADER]: Becoming Leader. State: Replica: 6142819da9d946988989c222bba8bebd, State: Running, Role: LEADER
I20260809 05:01:38.379045 32487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6142819da9d946988989c222bba8bebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46465 } }
I20260809 05:01:38.382292 32490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6142819da9d946988989c222bba8bebd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6142819da9d946988989c222bba8bebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46465 } } }
I20260809 05:01:38.382380 32491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6142819da9d946988989c222bba8bebd. Latest consensus state: current_term: 1 leader_uuid: "6142819da9d946988989c222bba8bebd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6142819da9d946988989c222bba8bebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46465 } } }
I20260809 05:01:38.382717 32490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:38.382751 32491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:38.386405 32478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:38.368122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:42449 (local address 127.0.0.1:46465)
0809 05:01:38.368311 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:38.368318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:38.368756 (+ 438us) server_negotiation.cc:407] Connection header received
0809 05:01:38.368981 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:38.368986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:38.369047 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:38.369141 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:38.370998 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:38.371866 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:38.373571 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:38.374292 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:38.374433 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:38.378091 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:38.378099 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:38.378104 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:38.378146 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:38.382026 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:38.383263 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:38.383288 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:38.383292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:38.383532 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:38.384149 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:38.384154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:38.384157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:38.384596 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:38.384881 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:38.385995 (+ 1114us) server_negotiation.cc:299] Negotiation successful
0809 05:01:38.386126 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 05:01:38.391539 32507 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:01:38.391662 32507 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:01:38.391772 32501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:38.393901 32501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:38.402601 32501 catalog_manager.cc:1347] Generated new cluster ID: d5e465dafb8943458fc5d992112ab5ca
I20260809 05:01:38.402702 32501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:38.439803 32501 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:38.441144 32501 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:38.449527 32501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6142819da9d946988989c222bba8bebd: Generated new TSK 0
I20260809 05:01:38.450563 32501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:38.468926 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32360
I20260809 05:01:38.470291 32462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:38.564145 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 32360
W20260809 05:01:38.590248 32336 connection.cc:535] client connection to 127.0.0.1:46465 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46465): Connection reset by peer (error 104)
2026-08-09T05:01:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2232 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:38 dist-test-slave-21b9 krb5kdc[32515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:38 dist-test-slave-21b9 krb5kdc[32515](info): set up 2 sockets
Aug 09 05:01:38 dist-test-slave-21b9 krb5kdc[32515](info): commencing operation
krb5kdc: starting...
W20260809 05:01:39.813397 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:40 dist-test-slave-21b9 krb5kdc[32515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:40.158954 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:40.503242 32531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:40.503598 32531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:40.503734 32531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:40.510977 32531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:40.511085 32531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:40.511162 32531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:40.511238 32531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:40.511307 32531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:40.521198 32531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40835
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:40.522516 32531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:40.523871 32531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:40.535254 32544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:40.535375 32540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:40.535375 32541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:40.536207 32531 server_base.cc:1034] running on GCE node
I20260809 05:01:40.537016 32531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:40.538388 32531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:40.539594 32531 hybrid_clock.cc:648] HybridClock initialized: now 1786251700539540 us; error 49 us; skew 500 ppm
I20260809 05:01:40.539921 32531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:40 dist-test-slave-21b9 krb5kdc[32515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251700, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:40.542919 32531 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:40.544394 32531 webserver.cc:460] Webserver started at http://127.0.0.1:35073/ using document root <none> and password file <none>
I20260809 05:01:40.545066 32531 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:40.545177 32531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:40.545440 32531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:40.548401 32531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "a309afeacfd9428ea55ed6befdb899e7"
format_stamp: "Formatted at 2026-08-09 05:01:40 on dist-test-slave-21b9"
server_key: "91b4c60d4ba167e983fa52161348b058"
server_key_iv: "1f8c851def4fcb22ddfc63837e11e976"
server_key_version: "encryptionkey@0"
I20260809 05:01:40.549152 32531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "a309afeacfd9428ea55ed6befdb899e7"
format_stamp: "Formatted at 2026-08-09 05:01:40 on dist-test-slave-21b9"
server_key: "91b4c60d4ba167e983fa52161348b058"
server_key_iv: "1f8c851def4fcb22ddfc63837e11e976"
server_key_version: "encryptionkey@0"
I20260809 05:01:40.554307 32531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:01:40.557888 32556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:40.559376 32531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:40.559543 32531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "a309afeacfd9428ea55ed6befdb899e7"
format_stamp: "Formatted at 2026-08-09 05:01:40 on dist-test-slave-21b9"
server_key: "91b4c60d4ba167e983fa52161348b058"
server_key_iv: "1f8c851def4fcb22ddfc63837e11e976"
server_key_version: "encryptionkey@0"
I20260809 05:01:40.559712 32531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:40.581665 32531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:40.585579 32531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:40.585812 32531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:40.603488 32531 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40835
I20260809 05:01:40.603534 32651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40835 every 8 connection(s)
I20260809 05:01:40.604903 32531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:40.608786 32653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:40.610934 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 32531
I20260809 05:01:40.611107 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:40.611647 30671 external_mini_cluster.cc:1428] Setting key bb9eec27618b4dc3a9d0783c39629a72
I20260809 05:01:40.622119 32653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7: Bootstrap starting.
I20260809 05:01:40.626214 32653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:40.626796 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:40.613478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:38221 (local address 127.0.0.1:40835)
0809 05:01:40.614253 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:40.614268 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:40.614328 (+ 60us) server_negotiation.cc:407] Connection header received
0809 05:01:40.616653 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:40.616683 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:40.617187 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:40.617624 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:40.618178 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:40.620278 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:40.621647 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:40.622638 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:40.622821 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:40.624028 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:40.624039 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:40.624058 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:40.624782 (+ 724us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:40.624883 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:40.624916 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:40.625078 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:40.625273 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 05:01:40.625459 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":286,"threads_started":1}
I20260809 05:01:40.627533 32653 log.cc:824] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:40.630533 32653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7: No bootstrap required, opened a new log
I20260809 05:01:40.630668 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:40.638106 32653 raft_consensus.cc:348] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a309afeacfd9428ea55ed6befdb899e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } }
I20260809 05:01:40.638463 32653 raft_consensus.cc:374] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:40.638567 32653 raft_consensus.cc:729] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a309afeacfd9428ea55ed6befdb899e7, State: Initialized, Role: FOLLOWER
I20260809 05:01:40.639190 32653 consensus_queue.cc:260] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a309afeacfd9428ea55ed6befdb899e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } }
I20260809 05:01:40.639498 32653 raft_consensus.cc:388] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:40.639640 32653 raft_consensus.cc:482] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:40.639806 32653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:40.640852 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:40.634797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:43951 (local address 127.0.0.1:40835)
0809 05:01:40.634950 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:40.634956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:40.635422 (+ 466us) server_negotiation.cc:407] Connection header received
0809 05:01:40.635599 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:40.635602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:40.635666 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:40.635747 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:40.636230 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:40.637023 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:40.638264 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:40.638801 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:40.638937 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:40.640104 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:40.640108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:40.640114 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:40.640131 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:40.640183 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:40.640197 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:40.640278 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:40.640493 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 05:01:40.640534 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 05:01:40.642503 32653 raft_consensus.cc:504] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a309afeacfd9428ea55ed6befdb899e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } }
I20260809 05:01:40.642976 32653 leader_election.cc:302] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a309afeacfd9428ea55ed6befdb899e7; no voters:
I20260809 05:01:40.643528 32653 leader_election.cc:288] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:40.643620 32667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:40.644277 32667 raft_consensus.cc:686] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [term 1 LEADER]: Becoming Leader. State: Replica: a309afeacfd9428ea55ed6befdb899e7, State: Running, Role: LEADER
I20260809 05:01:40.644863 32667 consensus_queue.cc:237] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a309afeacfd9428ea55ed6befdb899e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } }
I20260809 05:01:40.645501 32653 sys_catalog.cc:564] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:40.647571 32670 sys_catalog.cc:455] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a309afeacfd9428ea55ed6befdb899e7. Latest consensus state: current_term: 1 leader_uuid: "a309afeacfd9428ea55ed6befdb899e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a309afeacfd9428ea55ed6befdb899e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } } }
I20260809 05:01:40.647944 32670 sys_catalog.cc:458] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:40.648375 32669 sys_catalog.cc:455] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a309afeacfd9428ea55ed6befdb899e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a309afeacfd9428ea55ed6befdb899e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } } }
I20260809 05:01:40.648684 32669 sys_catalog.cc:458] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:40.650312 32676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:40.655956 32676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:40.665618 32676 catalog_manager.cc:1347] Generated new cluster ID: c5cfd00437a245fd8706b4d9180611fc
I20260809 05:01:40.665774 32676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:40.686740 32676 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:40.688675 32676 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:40.702194 32676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a309afeacfd9428ea55ed6befdb899e7: Generated new TSK 0
I20260809 05:01:40.703222 32676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:40.713259 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32531
W20260809 05:01:40.715054 32646 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.177:38221
W20260809 05:01:40.715802 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32531: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:40.715973 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 32531
W20260809 05:01:40.749312 32513 connection.cc:535] client connection to 127.0.0.1:40835 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40835): Connection reset by peer (error 104)
2026-08-09T05:01:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2158 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:40 dist-test-slave-21b9 krb5kdc[32694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:40 dist-test-slave-21b9 krb5kdc[32694](info): set up 2 sockets
Aug 09 05:01:40 dist-test-slave-21b9 krb5kdc[32694](info): commencing operation
krb5kdc: starting...
W20260809 05:01:41.962428 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:42 dist-test-slave-21b9 krb5kdc[32694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:01:42.279489 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45247
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:42.629122 32724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:42.629431 32724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:42.629540 32724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:42.636659 32724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:42.636775 32724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:42.636834 32724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:42.636902 32724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:42.636953 32724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:42.646708 32724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45247
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:42.648029 32724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:42.649435 32724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:42.661257 32731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:42.661317 32732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:42.661339 32736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:42.662168 32724 server_base.cc:1034] running on GCE node
I20260809 05:01:42.662891 32724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:42.664312 32724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:42.665525 32724 hybrid_clock.cc:648] HybridClock initialized: now 1786251702665476 us; error 41 us; skew 500 ppm
I20260809 05:01:42.665846 32724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:42 dist-test-slave-21b9 krb5kdc[32694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251702, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:42.668872 32724 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:01:42.670297 32724 webserver.cc:460] Webserver started at http://127.0.0.1:39223/ using document root <none> and password file <none>
I20260809 05:01:42.670972 32724 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:42.671095 32724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:42.671458 32724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:42.674466 32724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "8cd68427b38c4f9db46429d0855f35dc"
format_stamp: "Formatted at 2026-08-09 05:01:42 on dist-test-slave-21b9"
server_key: "e9e268b7f67a9abb4425e534e44a8d7b"
server_key_iv: "26d49589cad66850701ca1b3d8946425"
server_key_version: "encryptionkey@0"
I20260809 05:01:42.675243 32724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "8cd68427b38c4f9db46429d0855f35dc"
format_stamp: "Formatted at 2026-08-09 05:01:42 on dist-test-slave-21b9"
server_key: "e9e268b7f67a9abb4425e534e44a8d7b"
server_key_iv: "26d49589cad66850701ca1b3d8946425"
server_key_version: "encryptionkey@0"
I20260809 05:01:42.681201 32724 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 05:01:42.684719 32748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:42.686007 32724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 05:01:42.686170 32724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "8cd68427b38c4f9db46429d0855f35dc"
format_stamp: "Formatted at 2026-08-09 05:01:42 on dist-test-slave-21b9"
server_key: "e9e268b7f67a9abb4425e534e44a8d7b"
server_key_iv: "26d49589cad66850701ca1b3d8946425"
server_key_version: "encryptionkey@0"
I20260809 05:01:42.686367 32724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:42.692032 32724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:42.696113 32724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:42.696391 32724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:42.714202 32724 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45247
I20260809 05:01:42.714250 356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45247 every 8 connection(s)
I20260809 05:01:42.715615 32724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:42.719353 358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:42.720714 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 32724
I20260809 05:01:42.720906 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:42.721473 30671 external_mini_cluster.cc:1428] Setting key c3c8429ddc50b0916e0fcf1ece60a751
I20260809 05:01:42.729034 364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:42.723320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:43771 (local address 127.0.0.1:45247)
0809 05:01:42.724106 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:42.724118 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:42.724160 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:01:42.725736 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:42.725762 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:42.726110 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:42.726463 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:42.727499 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:42.727506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:42.727519 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:42.727534 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:42.727595 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:42.727619 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:42.727823 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:42.727954 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 05:01:42.728152 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":304,"threads_started":1}
I20260809 05:01:42.732887 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:42.732959 358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc: Bootstrap starting.
I20260809 05:01:42.736021 358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:42.737211 358 log.cc:824] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:42.739341 364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:42.736950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:34155 (local address 127.0.0.1:45247)
0809 05:01:42.737088 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:42.737092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:42.737577 (+ 485us) server_negotiation.cc:407] Connection header received
0809 05:01:42.737740 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:42.737744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:42.737789 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:42.737881 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:42.738713 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:42.738719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:42.738723 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:42.738742 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:42.738778 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:42.738793 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:42.738886 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:42.738998 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 05:01:42.739061 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 05:01:42.740147 358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc: No bootstrap required, opened a new log
I20260809 05:01:42.747670 358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd68427b38c4f9db46429d0855f35dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45247 } }
I20260809 05:01:42.748014 358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:42.748087 358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cd68427b38c4f9db46429d0855f35dc, State: Initialized, Role: FOLLOWER
I20260809 05:01:42.748616 358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd68427b38c4f9db46429d0855f35dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45247 } }
I20260809 05:01:42.748826 358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:42.748932 358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:42.749056 358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:42.751204 358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd68427b38c4f9db46429d0855f35dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45247 } }
I20260809 05:01:42.751693 358 leader_election.cc:302] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cd68427b38c4f9db46429d0855f35dc; no voters:
I20260809 05:01:42.752233 358 leader_election.cc:288] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:42.752324 374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:42.753742 374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [term 1 LEADER]: Becoming Leader. State: Replica: 8cd68427b38c4f9db46429d0855f35dc, State: Running, Role: LEADER
I20260809 05:01:42.754235 374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd68427b38c4f9db46429d0855f35dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45247 } }
I20260809 05:01:42.754755 358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:42.757230 377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cd68427b38c4f9db46429d0855f35dc. Latest consensus state: current_term: 1 leader_uuid: "8cd68427b38c4f9db46429d0855f35dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd68427b38c4f9db46429d0855f35dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45247 } } }
I20260809 05:01:42.757558 375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cd68427b38c4f9db46429d0855f35dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd68427b38c4f9db46429d0855f35dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45247 } } }
I20260809 05:01:42.757756 377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:42.757822 375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:42.758635 387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:42.763542 387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:42.772500 387 catalog_manager.cc:1347] Generated new cluster ID: d8a5e64585894216931414b0ea8269bd
I20260809 05:01:42.772612 387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:42.792579 387 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:42.793005 387 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:42.799458 387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cd68427b38c4f9db46429d0855f35dc: Generated new TSK 0
I20260809 05:01:42.800215 387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:42.806953 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32724
W20260809 05:01:42.808183 348 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.177:43771
W20260809 05:01:42.808830 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 32724: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:42.808956 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 32724
2026-08-09T05:01:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2087 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:42 dist-test-slave-21b9 krb5kdc[407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:42 dist-test-slave-21b9 krb5kdc[407](info): set up 2 sockets
Aug 09 05:01:42 dist-test-slave-21b9 krb5kdc[407](info): commencing operation
krb5kdc: starting...
W20260809 05:01:44.054224 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:44 dist-test-slave-21b9 krb5kdc[407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 05:01:44.364763 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35119
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:35119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:44.705760 428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:44.706090 428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:44.706233 428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:44.713419 428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:44.713536 428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:44.713614 428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:44.713697 428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:44.713758 428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:44.723456 428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:35119
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:35119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:44.724769 428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:44.726100 428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:44.737628 438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:44.737522 441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:44.737661 445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:44.738621 428 server_base.cc:1034] running on GCE node
I20260809 05:01:44.739427 428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:44.740741 428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:44.741926 428 hybrid_clock.cc:648] HybridClock initialized: now 1786251704741872 us; error 47 us; skew 500 ppm
I20260809 05:01:44.742271 428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:44 dist-test-slave-21b9 krb5kdc[407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251704, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:44.745283 428 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:01:44.746625 428 webserver.cc:460] Webserver started at http://127.29.243.254:37053/ using document root <none> and password file <none>
I20260809 05:01:44.747262 428 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:44.747408 428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:44.747674 428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:44.750473 428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "c8f4306c3a4447f3ba1724e0115b5159"
format_stamp: "Formatted at 2026-08-09 05:01:44 on dist-test-slave-21b9"
server_key: "a3f030743eaa7840533cdcc328f0b22d"
server_key_iv: "b2079f3ff350bda4c393b0e39819b928"
server_key_version: "encryptionkey@0"
I20260809 05:01:44.751174 428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "c8f4306c3a4447f3ba1724e0115b5159"
format_stamp: "Formatted at 2026-08-09 05:01:44 on dist-test-slave-21b9"
server_key: "a3f030743eaa7840533cdcc328f0b22d"
server_key_iv: "b2079f3ff350bda4c393b0e39819b928"
server_key_version: "encryptionkey@0"
I20260809 05:01:44.756212 428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 05:01:44.759641 456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:44.760890 428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:01:44.761066 428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "c8f4306c3a4447f3ba1724e0115b5159"
format_stamp: "Formatted at 2026-08-09 05:01:44 on dist-test-slave-21b9"
server_key: "a3f030743eaa7840533cdcc328f0b22d"
server_key_iv: "b2079f3ff350bda4c393b0e39819b928"
server_key_version: "encryptionkey@0"
I20260809 05:01:44.761240 428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:44.776162 428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:44.780453 428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:44.780714 428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:44.797547 428 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:35119
I20260809 05:01:44.797580 531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:35119 every 8 connection(s)
I20260809 05:01:44.798840 428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:44.802611 532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:44.804392 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 428
I20260809 05:01:44.804565 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:44.805004 30671 external_mini_cluster.cc:1428] Setting key 89da1a5e1480526a7916f6e902da9807
I20260809 05:01:44.814404 532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159: Bootstrap starting.
I20260809 05:01:44.818235 532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:44.819485 532 log.cc:824] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159: Log is configured to *not* fsync() on all Append() calls
Aug 09 05:01:44 dist-test-slave-21b9 krb5kdc[407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:44.822384 532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159: No bootstrap required, opened a new log
I20260809 05:01:44.828326 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:44.806852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46377 (local address 127.29.243.254:35119)
0809 05:01:44.807618 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:44.807629 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:44.807676 (+ 47us) server_negotiation.cc:407] Connection header received
0809 05:01:44.809358 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:44.809380 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:44.809750 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:44.810063 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:44.811856 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:44.813180 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:44.814862 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:44.815563 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:44.815785 (+ 222us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:44.820869 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:44.820878 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:44.820893 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:44.820934 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:44.823988 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:44.824799 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:44.824806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:44.824812 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:44.824992 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:44.825490 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:44.825495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:44.825497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:44.826069 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:44.826397 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:44.826923 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 05:01:44.827237 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":333,"threads_started":1}
I20260809 05:01:44.830293 532 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f4306c3a4447f3ba1724e0115b5159" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35119 } }
I20260809 05:01:44.830652 532 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:44.830771 532 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8f4306c3a4447f3ba1724e0115b5159, State: Initialized, Role: FOLLOWER
I20260809 05:01:44.831369 532 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f4306c3a4447f3ba1724e0115b5159" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35119 } }
I20260809 05:01:44.831661 532 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:44.831810 532 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:44.831997 532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:44.832144 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:44.834091 532 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f4306c3a4447f3ba1724e0115b5159" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35119 } }
I20260809 05:01:44.834640 532 leader_election.cc:302] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8f4306c3a4447f3ba1724e0115b5159; no voters:
I20260809 05:01:44.835238 532 leader_election.cc:288] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:44.835372 547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:44.838598 532 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:44.839591 547 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [term 1 LEADER]: Becoming Leader. State: Replica: c8f4306c3a4447f3ba1724e0115b5159, State: Running, Role: LEADER
I20260809 05:01:44.840306 547 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f4306c3a4447f3ba1724e0115b5159" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35119 } }
I20260809 05:01:44.850075 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:44.836307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56643 (local address 127.29.243.254:35119)
0809 05:01:44.836567 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:44.836572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:44.836861 (+ 289us) server_negotiation.cc:407] Connection header received
0809 05:01:44.837151 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:44.837156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:44.837238 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:44.837301 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:44.838851 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:44.839912 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:44.841487 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:44.841929 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:44.842035 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:44.844600 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:44.844607 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:44.844612 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:44.844661 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:44.847312 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:44.847995 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:44.848001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:44.848003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:44.848115 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:44.848557 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:44.848561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:44.848563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:44.848899 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:44.849167 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:44.849532 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 05:01:44.849699 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260809 05:01:44.853220 552 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8f4306c3a4447f3ba1724e0115b5159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f4306c3a4447f3ba1724e0115b5159" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35119 } } }
I20260809 05:01:44.853132 551 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8f4306c3a4447f3ba1724e0115b5159. Latest consensus state: current_term: 1 leader_uuid: "c8f4306c3a4447f3ba1724e0115b5159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f4306c3a4447f3ba1724e0115b5159" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 35119 } } }
I20260809 05:01:44.853577 551 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:44.853574 552 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:44.854857 571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:44.860008 571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:44.868685 571 catalog_manager.cc:1347] Generated new cluster ID: 96367a9f66d24e11aa383d1314a20924
I20260809 05:01:44.868849 571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:44.876603 571 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:44.877830 571 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:44.891748 571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8f4306c3a4447f3ba1724e0115b5159: Generated new TSK 0
I20260809 05:01:44.892601 571 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:44.900676 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 428
I20260809 05:01:44.901854 522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:44.990574 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 428
W20260809 05:01:45.012737 402 connection.cc:535] client connection to 127.29.243.254:35119 recv error: Network error: failed to read from TLS socket (remote: 127.29.243.254:35119): Connection reset by peer (error 104)
2026-08-09T05:01:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2175 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:45 dist-test-slave-21b9 krb5kdc[586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:45 dist-test-slave-21b9 krb5kdc[586](info): set up 2 sockets
Aug 09 05:01:45 dist-test-slave-21b9 krb5kdc[586](info): commencing operation
krb5kdc: starting...
W20260809 05:01:46.238958 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:46 dist-test-slave-21b9 krb5kdc[586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 05:01:46.563122 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:38815
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:38815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:46.908666 613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:46.908968 613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:46.909127 613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:46.916227 613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:46.916349 613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:46.916440 613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:46.916523 613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:46.916589 613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:46.926147 613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:38815
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:38815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:46.927526 613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:46.928869 613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:46.940393 621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:46.940430 625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:46.940469 622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:46.941656 613 server_base.cc:1034] running on GCE node
I20260809 05:01:46.942423 613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:46.943985 613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:46.945206 613 hybrid_clock.cc:648] HybridClock initialized: now 1786251706945155 us; error 45 us; skew 500 ppm
I20260809 05:01:46.945560 613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:46 dist-test-slave-21b9 krb5kdc[586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251706, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:46.948583 613 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:01:46.949987 613 webserver.cc:460] Webserver started at http://127.29.243.254:43467/ using document root <none> and password file <none>
I20260809 05:01:46.950677 613 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:46.950796 613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:46.951114 613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:46.954134 613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "ed743b3d6f3f46c9bb87aee3f45bcfbe"
format_stamp: "Formatted at 2026-08-09 05:01:46 on dist-test-slave-21b9"
server_key: "394a26cae6ae2fdd09c0f668c295a552"
server_key_iv: "ce0aab81e4ad54780a248e04e94d67e9"
server_key_version: "encryptionkey@0"
I20260809 05:01:46.954876 613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "ed743b3d6f3f46c9bb87aee3f45bcfbe"
format_stamp: "Formatted at 2026-08-09 05:01:46 on dist-test-slave-21b9"
server_key: "394a26cae6ae2fdd09c0f668c295a552"
server_key_iv: "ce0aab81e4ad54780a248e04e94d67e9"
server_key_version: "encryptionkey@0"
I20260809 05:01:46.959887 613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 05:01:46.963315 634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:46.964696 613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:01:46.964869 613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "ed743b3d6f3f46c9bb87aee3f45bcfbe"
format_stamp: "Formatted at 2026-08-09 05:01:46 on dist-test-slave-21b9"
server_key: "394a26cae6ae2fdd09c0f668c295a552"
server_key_iv: "ce0aab81e4ad54780a248e04e94d67e9"
server_key_version: "encryptionkey@0"
I20260809 05:01:46.965059 613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:46.978943 613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:46.982852 613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:46.983117 613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:47.000211 613 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:38815
I20260809 05:01:47.000222 718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:38815 every 8 connection(s)
I20260809 05:01:47.001562 613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:47.004613 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 613
I20260809 05:01:47.004779 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:47.005194 30671 external_mini_cluster.cc:1428] Setting key 13600ce0cc8405f723eadc42e8bf8f78
I20260809 05:01:47.005721 719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:47.019848 722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:47.007029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51701 (local address 127.29.243.254:38815)
0809 05:01:47.007868 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:47.007884 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:47.007933 (+ 49us) server_negotiation.cc:407] Connection header received
0809 05:01:47.009596 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:47.009625 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:47.010153 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:47.010604 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:47.011119 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:47.013153 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:47.014449 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:47.015492 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:47.015670 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:47.016901 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:47.016912 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:47.016932 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:47.017712 (+ 780us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:47.017813 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:47.017846 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:47.018017 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:47.018252 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 05:01:47.018463 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":349,"threads_started":1}
I20260809 05:01:47.021257 719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe: Bootstrap starting.
I20260809 05:01:47.023795 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:47.024605 719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:47.025596 719 log.cc:824] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:47.028734 719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe: No bootstrap required, opened a new log
I20260809 05:01:47.034736 722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:47.028266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56003 (local address 127.29.243.254:38815)
0809 05:01:47.028492 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:47.028497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:47.028804 (+ 307us) server_negotiation.cc:407] Connection header received
0809 05:01:47.028981 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:47.028986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:47.029074 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:47.029188 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:47.029639 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:47.030777 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:47.032115 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:47.032796 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:47.032931 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:47.033980 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:47.033989 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:47.033994 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:47.034016 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:47.034066 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:47.034085 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:47.034194 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:47.034327 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 05:01:47.034391 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 05:01:47.037863 719 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed743b3d6f3f46c9bb87aee3f45bcfbe" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38815 } }
I20260809 05:01:47.038220 719 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:47.038293 719 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed743b3d6f3f46c9bb87aee3f45bcfbe, State: Initialized, Role: FOLLOWER
I20260809 05:01:47.038812 719 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed743b3d6f3f46c9bb87aee3f45bcfbe" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38815 } }
I20260809 05:01:47.039017 719 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:47.039119 719 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:47.039247 719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:47.041435 719 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed743b3d6f3f46c9bb87aee3f45bcfbe" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38815 } }
I20260809 05:01:47.041963 719 leader_election.cc:302] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed743b3d6f3f46c9bb87aee3f45bcfbe; no voters:
I20260809 05:01:47.042577 719 leader_election.cc:288] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:47.042738 739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:47.043484 739 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [term 1 LEADER]: Becoming Leader. State: Replica: ed743b3d6f3f46c9bb87aee3f45bcfbe, State: Running, Role: LEADER
I20260809 05:01:47.044013 739 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed743b3d6f3f46c9bb87aee3f45bcfbe" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38815 } }
I20260809 05:01:47.044720 719 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:47.046857 740 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed743b3d6f3f46c9bb87aee3f45bcfbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed743b3d6f3f46c9bb87aee3f45bcfbe" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38815 } } }
I20260809 05:01:47.046886 741 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed743b3d6f3f46c9bb87aee3f45bcfbe. Latest consensus state: current_term: 1 leader_uuid: "ed743b3d6f3f46c9bb87aee3f45bcfbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed743b3d6f3f46c9bb87aee3f45bcfbe" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38815 } } }
I20260809 05:01:47.047429 741 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:47.047880 740 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:47.048971 752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:47.054248 752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:47.062865 752 catalog_manager.cc:1347] Generated new cluster ID: 41939ad705d14775998201910f01cbcf
I20260809 05:01:47.062986 752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:47.096873 752 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:47.098824 752 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:47.142328 752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed743b3d6f3f46c9bb87aee3f45bcfbe: Generated new TSK 0
I20260809 05:01:47.143290 752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:47.157603 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 613
W20260809 05:01:47.158768 712 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51701
W20260809 05:01:47.159624 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 613: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:47.159760 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 613
W20260809 05:01:47.185467 583 connection.cc:535] client connection to 127.29.243.254:38815 recv error: Network error: failed to read from TLS socket (remote: 127.29.243.254:38815): Connection reset by peer (error 104)
2026-08-09T05:01:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2172 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:47 dist-test-slave-21b9 krb5kdc[775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:47 dist-test-slave-21b9 krb5kdc[775](info): set up 2 sockets
Aug 09 05:01:47 dist-test-slave-21b9 krb5kdc[775](info): commencing operation
krb5kdc: starting...
W20260809 05:01:48.399143 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:48 dist-test-slave-21b9 krb5kdc[775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 05:01:48.723165 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:43865
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:35719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:43865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:49.060141 801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:49.060437 801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:49.060575 801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:49.067608 801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:49.067723 801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:49.067791 801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:49.067880 801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:49.067941 801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:49.077827 801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:35719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:43865
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:43865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:49.079170 801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:49.080554 801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:49.091964 815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:49.092109 812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:49.091985 813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:49.092736 801 server_base.cc:1034] running on GCE node
I20260809 05:01:49.093487 801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:49.094816 801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:49.096043 801 hybrid_clock.cc:648] HybridClock initialized: now 1786251709095981 us; error 40 us; skew 500 ppm
I20260809 05:01:49.096457 801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:49 dist-test-slave-21b9 krb5kdc[775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251709, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:49.099613 801 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:01:49.101042 801 webserver.cc:460] Webserver started at http://127.29.243.254:44379/ using document root <none> and password file <none>
I20260809 05:01:49.101668 801 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:49.101766 801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:49.102030 801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:49.104952 801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "a4f1d3d5d86149ab914c4c2836c31da5"
format_stamp: "Formatted at 2026-08-09 05:01:49 on dist-test-slave-21b9"
server_key: "998ecdef8f3200ee385a5cb8da50029a"
server_key_iv: "918d24639f9bf4cfdac518938ae48691"
server_key_version: "encryptionkey@0"
I20260809 05:01:49.105633 801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "a4f1d3d5d86149ab914c4c2836c31da5"
format_stamp: "Formatted at 2026-08-09 05:01:49 on dist-test-slave-21b9"
server_key: "998ecdef8f3200ee385a5cb8da50029a"
server_key_iv: "918d24639f9bf4cfdac518938ae48691"
server_key_version: "encryptionkey@0"
I20260809 05:01:49.110543 801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 05:01:49.113893 824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:49.115233 801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:01:49.115439 801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "a4f1d3d5d86149ab914c4c2836c31da5"
format_stamp: "Formatted at 2026-08-09 05:01:49 on dist-test-slave-21b9"
server_key: "998ecdef8f3200ee385a5cb8da50029a"
server_key_iv: "918d24639f9bf4cfdac518938ae48691"
server_key_version: "encryptionkey@0"
I20260809 05:01:49.115608 801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:49.120959 801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:49.124992 801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:49.125231 801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:49.142540 801 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:43865
I20260809 05:01:49.142578 907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:43865 every 8 connection(s)
I20260809 05:01:49.143975 801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:49.147833 908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:49.154259 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 801
I20260809 05:01:49.154429 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:49.154855 30671 external_mini_cluster.cc:1428] Setting key b3a4e7c5a5182ac412707692f07a28b0
I20260809 05:01:49.158427 908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5: Bootstrap starting.
I20260809 05:01:49.161993 908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:49.162420 912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:49.156634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45141 (local address 127.29.243.254:43865)
0809 05:01:49.157422 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:49.157435 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:49.157477 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:01:49.159111 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:49.159134 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:49.159508 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:49.159820 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:49.160892 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:49.160900 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:49.160913 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:49.160928 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:49.160988 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:49.161008 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:49.161175 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:49.161271 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 05:01:49.161452 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":335,"threads_started":1}
I20260809 05:01:49.163242 908 log.cc:824] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:49.166604 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:49.166651 908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5: No bootstrap required, opened a new log
I20260809 05:01:49.173620 912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:49.170848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46931 (local address 127.29.243.254:43865)
0809 05:01:49.171034 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:49.171042 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:49.171771 (+ 729us) server_negotiation.cc:407] Connection header received
0809 05:01:49.171908 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:49.171914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:49.171988 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:49.172094 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:49.172890 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:49.172896 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:49.172900 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:49.172918 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:49.172977 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:49.172998 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:49.173132 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:49.173288 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 05:01:49.173344 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 05:01:49.174824 908 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f1d3d5d86149ab914c4c2836c31da5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43865 } }
I20260809 05:01:49.175179 908 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:49.175283 908 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f1d3d5d86149ab914c4c2836c31da5, State: Initialized, Role: FOLLOWER
I20260809 05:01:49.175865 908 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f1d3d5d86149ab914c4c2836c31da5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43865 } }
I20260809 05:01:49.176096 908 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:49.176218 908 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:49.176380 908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:49.178504 908 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f1d3d5d86149ab914c4c2836c31da5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43865 } }
I20260809 05:01:49.179045 908 leader_election.cc:302] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4f1d3d5d86149ab914c4c2836c31da5; no voters:
I20260809 05:01:49.179623 908 leader_election.cc:288] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:49.179735 925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:49.180413 925 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [term 1 LEADER]: Becoming Leader. State: Replica: a4f1d3d5d86149ab914c4c2836c31da5, State: Running, Role: LEADER
I20260809 05:01:49.180938 925 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f1d3d5d86149ab914c4c2836c31da5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43865 } }
I20260809 05:01:49.181689 908 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:49.183874 928 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4f1d3d5d86149ab914c4c2836c31da5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f1d3d5d86149ab914c4c2836c31da5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43865 } } }
I20260809 05:01:49.183888 933 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4f1d3d5d86149ab914c4c2836c31da5. Latest consensus state: current_term: 1 leader_uuid: "a4f1d3d5d86149ab914c4c2836c31da5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f1d3d5d86149ab914c4c2836c31da5" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 43865 } } }
I20260809 05:01:49.184387 928 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:49.184387 933 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:49.191234 943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:49.196907 943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:49.205693 943 catalog_manager.cc:1347] Generated new cluster ID: 758d81484a294ca0adce251be316f994
I20260809 05:01:49.205808 943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:49.233585 943 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:49.234174 943 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:49.249754 943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4f1d3d5d86149ab914c4c2836c31da5: Generated new TSK 0
I20260809 05:01:49.250698 943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:49.261090 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 801
W20260809 05:01:49.262208 903 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45141
W20260809 05:01:49.262872 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 801: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:49.263031 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 801
2026-08-09T05:01:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2108 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:49 dist-test-slave-21b9 krb5kdc[966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:49 dist-test-slave-21b9 krb5kdc[966](info): set up 2 sockets
Aug 09 05:01:49 dist-test-slave-21b9 krb5kdc[966](info): commencing operation
krb5kdc: starting...
W20260809 05:01:50.522293 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:50 dist-test-slave-21b9 krb5kdc[966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 05:01:50.850178 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:39345
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:45359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:39345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:51.193276 988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:51.193588 988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:51.193733 988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:51.200726 988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:51.200853 988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:51.200933 988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:51.201135 988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:51.201256 988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:51.210979 988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:45359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:39345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:39345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:51.212342 988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:51.213665 988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:51.224916 998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:51.224933 1001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:51.224903 1004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:51.226019 988 server_base.cc:1034] running on GCE node
I20260809 05:01:51.226754 988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:51.228185 988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:51.229398 988 hybrid_clock.cc:648] HybridClock initialized: now 1786251711229343 us; error 38 us; skew 500 ppm
I20260809 05:01:51.229735 988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:51 dist-test-slave-21b9 krb5kdc[966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251711, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:51.232776 988 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:01:51.234153 988 webserver.cc:460] Webserver started at http://127.29.243.254:41219/ using document root <none> and password file <none>
I20260809 05:01:51.234830 988 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:51.234951 988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:51.235247 988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:51.238234 988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "0a51a5c34ec148059bf1511712b1eda1"
format_stamp: "Formatted at 2026-08-09 05:01:51 on dist-test-slave-21b9"
server_key: "d20d5f8ba913a3d8f869ac0aef4f95f9"
server_key_iv: "d5f13808f60b85038f19901cc6347c7c"
server_key_version: "encryptionkey@0"
I20260809 05:01:51.238967 988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "0a51a5c34ec148059bf1511712b1eda1"
format_stamp: "Formatted at 2026-08-09 05:01:51 on dist-test-slave-21b9"
server_key: "d20d5f8ba913a3d8f869ac0aef4f95f9"
server_key_iv: "d5f13808f60b85038f19901cc6347c7c"
server_key_version: "encryptionkey@0"
I20260809 05:01:51.244145 988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 05:01:51.247537 1012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:51.248833 988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:01:51.249011 988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "0a51a5c34ec148059bf1511712b1eda1"
format_stamp: "Formatted at 2026-08-09 05:01:51 on dist-test-slave-21b9"
server_key: "d20d5f8ba913a3d8f869ac0aef4f95f9"
server_key_iv: "d5f13808f60b85038f19901cc6347c7c"
server_key_version: "encryptionkey@0"
I20260809 05:01:51.249210 988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:51.277855 988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:51.281838 988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:51.282109 988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:51.298817 988 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:39345
I20260809 05:01:51.298848 1104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:39345 every 8 connection(s)
I20260809 05:01:51.300172 988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:51.302973 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 988
I20260809 05:01:51.303164 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:51.303750 30671 external_mini_cluster.cc:1428] Setting key f82775a1833989f2d2438620c565bfd3
I20260809 05:01:51.304462 1106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:01:51 dist-test-slave-21b9 krb5kdc[966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:51.318823 1106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1: Bootstrap starting.
I20260809 05:01:51.322141 1106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:51.323422 1106 log.cc:824] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:51.326733 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:51.305574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58787 (local address 127.29.243.254:39345)
0809 05:01:51.306333 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:51.306344 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:51.306396 (+ 52us) server_negotiation.cc:407] Connection header received
0809 05:01:51.308017 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:51.308039 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:51.308430 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:51.308751 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:51.310523 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:51.311967 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:51.313729 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:51.314422 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:51.314570 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:51.319518 (+ 4948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:51.319527 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:51.319542 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:51.319578 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:51.322501 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:51.323310 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:51.323319 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:51.323328 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:51.323503 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:51.323921 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:51.323925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:51.323928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:51.324684 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:51.324959 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:51.325372 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 05:01:51.325700 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":322,"threads_started":1}
I20260809 05:01:51.326712 1106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1: No bootstrap required, opened a new log
I20260809 05:01:51.330611 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:51.336580 1106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a51a5c34ec148059bf1511712b1eda1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39345 } }
I20260809 05:01:51.337031 1106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:51.337129 1106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a51a5c34ec148059bf1511712b1eda1, State: Initialized, Role: FOLLOWER
I20260809 05:01:51.337702 1106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a51a5c34ec148059bf1511712b1eda1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39345 } }
I20260809 05:01:51.337916 1106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:51.338047 1106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:51.338192 1106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:51.340868 1106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a51a5c34ec148059bf1511712b1eda1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39345 } }
I20260809 05:01:51.341526 1106 leader_election.cc:302] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a51a5c34ec148059bf1511712b1eda1; no voters:
I20260809 05:01:51.342183 1106 leader_election.cc:288] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:51.342307 1124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:51.343472 1124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [term 1 LEADER]: Becoming Leader. State: Replica: 0a51a5c34ec148059bf1511712b1eda1, State: Running, Role: LEADER
I20260809 05:01:51.344107 1124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a51a5c34ec148059bf1511712b1eda1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39345 } }
I20260809 05:01:51.344596 1106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:51.347476 1126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a51a5c34ec148059bf1511712b1eda1. Latest consensus state: current_term: 1 leader_uuid: "0a51a5c34ec148059bf1511712b1eda1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a51a5c34ec148059bf1511712b1eda1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39345 } } }
I20260809 05:01:51.347870 1126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:51.348421 1125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a51a5c34ec148059bf1511712b1eda1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a51a5c34ec148059bf1511712b1eda1" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39345 } } }
I20260809 05:01:51.348696 1125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:51.351774 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:51.335839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45071 (local address 127.29.243.254:39345)
0809 05:01:51.336013 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:51.336019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:51.336451 (+ 432us) server_negotiation.cc:407] Connection header received
0809 05:01:51.336586 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:51.336590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:51.336669 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:51.336774 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:51.338381 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:51.339550 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:51.340754 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:51.341484 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:51.341615 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:51.344263 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:51.344268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:51.344271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:51.344303 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:51.347593 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:51.348435 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:51.348442 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:51.348446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:51.348643 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:51.349166 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:51.349172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:51.349176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:51.349591 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:51.349879 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:51.351047 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0809 05:01:51.351300 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 05:01:51.358819 1135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:51.363883 1135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:51.372675 1135 catalog_manager.cc:1347] Generated new cluster ID: f1e0b33edcfa428a9764b55320bfb880
I20260809 05:01:51.372788 1135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:51.427438 1135 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:51.429350 1135 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:51.440160 1135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a51a5c34ec148059bf1511712b1eda1: Generated new TSK 0
I20260809 05:01:51.440984 1135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:51.452555 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 988
I20260809 05:01:51.453653 1091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:01:51.547680 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 988
2026-08-09T05:01:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:51 dist-test-slave-21b9 krb5kdc[1161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:51 dist-test-slave-21b9 krb5kdc[1161](info): set up 2 sockets
Aug 09 05:01:51 dist-test-slave-21b9 krb5kdc[1161](info): commencing operation
krb5kdc: starting...
W20260809 05:01:52.790704 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:52 dist-test-slave-21b9 krb5kdc[1161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 05:01:53.116900 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:38797
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:37023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:38797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:53.460822 1187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:53.461118 1187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:53.461236 1187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:53.468371 1187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:53.468487 1187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:53.468557 1187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:53.468644 1187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:53.468698 1187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:53.478644 1187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:37023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:38797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:38797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:53.479983 1187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:53.481372 1187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:53.492686 1199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:53.492650 1197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:53.492738 1196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:53.493726 1187 server_base.cc:1034] running on GCE node
I20260809 05:01:53.494460 1187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:53.495823 1187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:53.497032 1187 hybrid_clock.cc:648] HybridClock initialized: now 1786251713496989 us; error 37 us; skew 500 ppm
I20260809 05:01:53.497361 1187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:53 dist-test-slave-21b9 krb5kdc[1161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251713, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:53.500365 1187 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:01:53.501798 1187 webserver.cc:460] Webserver started at http://127.29.243.254:43121/ using document root <none> and password file <none>
I20260809 05:01:53.502496 1187 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:53.502605 1187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:53.502900 1187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:53.505858 1187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "087b7a3d4f6847c79f36a9295fac91ce"
format_stamp: "Formatted at 2026-08-09 05:01:53 on dist-test-slave-21b9"
server_key: "f1d336616d3e7406c7fed7b146004a28"
server_key_iv: "3cee0cf6f93bb7e78a91550185dc9735"
server_key_version: "encryptionkey@0"
I20260809 05:01:53.506593 1187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "087b7a3d4f6847c79f36a9295fac91ce"
format_stamp: "Formatted at 2026-08-09 05:01:53 on dist-test-slave-21b9"
server_key: "f1d336616d3e7406c7fed7b146004a28"
server_key_iv: "3cee0cf6f93bb7e78a91550185dc9735"
server_key_version: "encryptionkey@0"
I20260809 05:01:53.511638 1187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:01:53.515105 1215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:53.516471 1187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:53.516618 1187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "087b7a3d4f6847c79f36a9295fac91ce"
format_stamp: "Formatted at 2026-08-09 05:01:53 on dist-test-slave-21b9"
server_key: "f1d336616d3e7406c7fed7b146004a28"
server_key_iv: "3cee0cf6f93bb7e78a91550185dc9735"
server_key_version: "encryptionkey@0"
I20260809 05:01:53.516798 1187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:53.537604 1187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:53.541805 1187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:53.542073 1187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:53.559185 1187 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:38797
I20260809 05:01:53.559217 1300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:38797 every 8 connection(s)
I20260809 05:01:53.560528 1187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:53.564323 1302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:53.568909 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 1187
I20260809 05:01:53.569130 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:53.569684 30671 external_mini_cluster.cc:1428] Setting key dbf91c4b47145e2cedd4fd9b6c2a6002
I20260809 05:01:53.575639 1302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce: Bootstrap starting.
I20260809 05:01:53.579172 1302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:53.580461 1302 log.cc:824] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:53.583220 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:53.571542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39065 (local address 127.29.243.254:38797)
0809 05:01:53.572337 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:53.572349 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:53.572394 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:01:53.574171 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:53.574196 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:53.574568 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:53.574867 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:53.575500 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:53.576839 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:53.578651 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:53.579397 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:53.579544 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:53.580750 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:53.580759 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:53.580773 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:53.581487 (+ 714us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:53.581605 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:53.581644 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:53.581771 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:53.581981 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 05:01:53.582233 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":366,"threads_started":1}
I20260809 05:01:53.583643 1302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce: No bootstrap required, opened a new log
I20260809 05:01:53.586886 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:53.591516 1302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087b7a3d4f6847c79f36a9295fac91ce" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38797 } }
I20260809 05:01:53.591940 1302 raft_consensus.cc:374] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:53.592059 1302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 087b7a3d4f6847c79f36a9295fac91ce, State: Initialized, Role: FOLLOWER
I20260809 05:01:53.592971 1302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087b7a3d4f6847c79f36a9295fac91ce" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38797 } }
I20260809 05:01:53.593220 1302 raft_consensus.cc:388] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:53.593330 1302 raft_consensus.cc:482] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:53.593495 1302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:53.596643 1302 raft_consensus.cc:504] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087b7a3d4f6847c79f36a9295fac91ce" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38797 } }
I20260809 05:01:53.597183 1320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:01:53.597209 1302 leader_election.cc:302] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 087b7a3d4f6847c79f36a9295fac91ce; no voters:
I20260809 05:01:53.597476 1320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087b7a3d4f6847c79f36a9295fac91ce" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38797 } }
I20260809 05:01:53.597890 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:53.591032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53249 (local address 127.29.243.254:38797)
0809 05:01:53.591210 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:53.591216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:53.591777 (+ 561us) server_negotiation.cc:407] Connection header received
0809 05:01:53.591949 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:53.591956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:53.592029 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:53.592120 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:53.592793 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:53.593736 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:53.595545 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:53.595972 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:53.596072 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:53.597176 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:53.597181 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:53.597184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:53.597204 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:53.597241 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:53.597256 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:53.597335 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:53.597473 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 05:01:53.597562 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 05:01:53.598742 1320 leader_election.cc:302] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 087b7a3d4f6847c79f36a9295fac91ce; no voters:
I20260809 05:01:53.599536 1320 leader_election.cc:288] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers
I20260809 05:01:53.599704 1325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:53.599915 1325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [term 1 LEADER]: Becoming Leader. State: Replica: 087b7a3d4f6847c79f36a9295fac91ce, State: Running, Role: LEADER
I20260809 05:01:53.600410 1325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087b7a3d4f6847c79f36a9295fac91ce" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38797 } }
I20260809 05:01:53.602221 1325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 087b7a3d4f6847c79f36a9295fac91ce. Latest consensus state: current_term: 1 leader_uuid: "087b7a3d4f6847c79f36a9295fac91ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087b7a3d4f6847c79f36a9295fac91ce" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38797 } } }
I20260809 05:01:53.602927 1325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:53.604092 1302 leader_election.cc:288] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:53.605051 1320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "087b7a3d4f6847c79f36a9295fac91ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087b7a3d4f6847c79f36a9295fac91ce" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 38797 } } }
I20260809 05:01:53.605263 1320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:53.605880 1302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:53.606091 1327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:53.611961 1327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:53.621287 1327 catalog_manager.cc:1347] Generated new cluster ID: 6d84442f204d49a4af711fc13da2ff8d
I20260809 05:01:53.621398 1327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:53.647099 1327 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:53.649085 1327 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:53.657094 1327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 087b7a3d4f6847c79f36a9295fac91ce: Generated new TSK 0
I20260809 05:01:53.658167 1327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:53.677088 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1187
W20260809 05:01:53.678329 1289 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39065
W20260809 05:01:53.679090 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1187: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:53.679242 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 1187
2026-08-09T05:01:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2133 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:53 dist-test-slave-21b9 krb5kdc[1350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:53 dist-test-slave-21b9 krb5kdc[1350](info): set up 2 sockets
Aug 09 05:01:53 dist-test-slave-21b9 krb5kdc[1350](info): commencing operation
krb5kdc: starting...
W20260809 05:01:54.933724 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:55 dist-test-slave-21b9 krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 05:01:55.246793 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:40165
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:40165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:55.598198 1371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:55.598519 1371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:55.598624 1371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:55.605865 1371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:55.606000 1371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:55.606053 1371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:55.606137 1371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:55.606187 1371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:55.616326 1371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:40165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:40165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:55.617697 1371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:55.619109 1371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:55.631253 1386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:55.631322 1383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:55.631749 1382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:55.632061 1371 server_base.cc:1034] running on GCE node
I20260809 05:01:55.632777 1371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:55.634109 1371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:55.635344 1371 hybrid_clock.cc:648] HybridClock initialized: now 1786251715635282 us; error 51 us; skew 500 ppm
I20260809 05:01:55.635686 1371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:55 dist-test-slave-21b9 krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251715, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:55.641764 1371 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:01:55.643200 1371 webserver.cc:460] Webserver started at http://127.29.243.254:41983/ using document root <none> and password file <none>
I20260809 05:01:55.643884 1371 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:55.644007 1371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:55.644296 1371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:55.647421 1371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "17b9814c5f0341f780311bde34b0320c"
format_stamp: "Formatted at 2026-08-09 05:01:55 on dist-test-slave-21b9"
server_key: "37ce7b408906d108929d527ac55bcc93"
server_key_iv: "05a6baab50f8f66dcd68dba1ae4fb232"
server_key_version: "encryptionkey@0"
I20260809 05:01:55.648164 1371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "17b9814c5f0341f780311bde34b0320c"
format_stamp: "Formatted at 2026-08-09 05:01:55 on dist-test-slave-21b9"
server_key: "37ce7b408906d108929d527ac55bcc93"
server_key_iv: "05a6baab50f8f66dcd68dba1ae4fb232"
server_key_version: "encryptionkey@0"
I20260809 05:01:55.653143 1371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:01:55.656487 1398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:55.657720 1371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:01:55.657869 1371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "17b9814c5f0341f780311bde34b0320c"
format_stamp: "Formatted at 2026-08-09 05:01:55 on dist-test-slave-21b9"
server_key: "37ce7b408906d108929d527ac55bcc93"
server_key_iv: "05a6baab50f8f66dcd68dba1ae4fb232"
server_key_version: "encryptionkey@0"
I20260809 05:01:55.658041 1371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:55.663625 1371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:55.667415 1371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:55.667673 1371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:55.685052 1371 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:40165
I20260809 05:01:55.685086 1481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:40165 every 8 connection(s)
I20260809 05:01:55.686420 1371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:55.687881 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 1371
I20260809 05:01:55.688045 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:55.688458 30671 external_mini_cluster.cc:1428] Setting key 1de4516aa32cfb22b8b77850ef71e6b9
I20260809 05:01:55.690589 1482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:55.695901 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:55.690149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36853 (local address 127.29.243.254:40165)
0809 05:01:55.690929 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:55.690940 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:55.690983 (+ 43us) server_negotiation.cc:407] Connection header received
0809 05:01:55.692576 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:55.692597 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:55.692935 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:55.693253 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:55.694292 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:55.694299 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:55.694312 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:55.694326 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:55.694435 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:55.694457 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:55.694627 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:55.694766 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 05:01:55.694937 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":386,"threads_started":1}
I20260809 05:01:55.699334 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:55.702582 1482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c: Bootstrap starting.
I20260809 05:01:55.706264 1482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:55.706821 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:55.704494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37989 (local address 127.29.243.254:40165)
0809 05:01:55.704678 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:55.704684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:55.705066 (+ 382us) server_negotiation.cc:407] Connection header received
0809 05:01:55.705183 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:55.705189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:55.705245 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:55.705348 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:55.706132 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:55.706137 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:55.706146 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:01:55.706164 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:55.706210 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:01:55.706230 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:55.706355 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:55.706459 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 05:01:55.706526 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 05:01:55.707558 1482 log.cc:824] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:55.710184 1482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c: No bootstrap required, opened a new log
I20260809 05:01:55.717317 1482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b9814c5f0341f780311bde34b0320c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40165 } }
I20260809 05:01:55.717689 1482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:01:55.717779 1482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17b9814c5f0341f780311bde34b0320c, State: Initialized, Role: FOLLOWER
I20260809 05:01:55.718324 1482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b9814c5f0341f780311bde34b0320c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40165 } }
I20260809 05:01:55.718541 1482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:01:55.718696 1482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:01:55.718847 1482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:55.722025 1482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b9814c5f0341f780311bde34b0320c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40165 } }
I20260809 05:01:55.722550 1482 leader_election.cc:302] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17b9814c5f0341f780311bde34b0320c; no voters:
I20260809 05:01:55.723107 1482 leader_election.cc:288] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:01:55.723235 1506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:55.723925 1506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [term 1 LEADER]: Becoming Leader. State: Replica: 17b9814c5f0341f780311bde34b0320c, State: Running, Role: LEADER
I20260809 05:01:55.724537 1506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b9814c5f0341f780311bde34b0320c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40165 } }
I20260809 05:01:55.725230 1482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:55.727478 1507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17b9814c5f0341f780311bde34b0320c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b9814c5f0341f780311bde34b0320c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40165 } } }
I20260809 05:01:55.727779 1507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:55.728170 1508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17b9814c5f0341f780311bde34b0320c. Latest consensus state: current_term: 1 leader_uuid: "17b9814c5f0341f780311bde34b0320c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b9814c5f0341f780311bde34b0320c" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 40165 } } }
I20260809 05:01:55.728487 1508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:55.729264 1516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:55.734803 1516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:01:55.743988 1516 catalog_manager.cc:1347] Generated new cluster ID: 46d5c7f1fdb44a12b3737326c2072ebc
I20260809 05:01:55.744100 1516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:55.762169 1516 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:55.762668 1516 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:55.769178 1516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17b9814c5f0341f780311bde34b0320c: Generated new TSK 0
I20260809 05:01:55.769934 1516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:55.779074 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1371
W20260809 05:01:55.780128 1471 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36853
W20260809 05:01:55.780778 30671 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1371: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:01:55.780915 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 1371
2026-08-09T05:01:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2097 ms)
[----------] 18 tests from AuthTokenIssuingTest (38798 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:01:55 dist-test-slave-21b9 krb5kdc[1533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:01:55 dist-test-slave-21b9 krb5kdc[1533](info): set up 2 sockets
Aug 09 05:01:55 dist-test-slave-21b9 krb5kdc[1533](info): commencing operation
krb5kdc: starting...
W20260809 05:01:57.044342 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.006s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:01:57 dist-test-slave-21b9 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:01:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:01:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 05:01:57.366446 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:39523
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:39523,127.29.243.253:35083,127.29.243.252:36705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:01:57.701373 1553 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:01:57.701710 1553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:57.701810 1553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:57.701938 1553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:57.709115 1553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:57.709196 1553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:57.709270 1553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:57.709347 1553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:57.709411 1553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:57.718992 1553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:39523,127.29.243.253:35083,127.29.243.252:36705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:39523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:57.720292 1553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:57.721637 1553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:57.733366 1561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:57.733315 1563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:57.733397 1567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:57.734402 1553 server_base.cc:1034] running on GCE node
I20260809 05:01:57.735167 1553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:57.736512 1553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:57.737730 1553 hybrid_clock.cc:648] HybridClock initialized: now 1786251717737664 us; error 40 us; skew 500 ppm
I20260809 05:01:57.738051 1553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:57 dist-test-slave-21b9 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251717, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:57.741098 1553 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:01:57.742535 1553 webserver.cc:460] Webserver started at http://127.29.243.254:44799/ using document root <none> and password file <none>
I20260809 05:01:57.743223 1553 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:57.743350 1553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:57.743655 1553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:57.746556 1553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "2275149ec8dd45e5b917da07c76bc384"
format_stamp: "Formatted at 2026-08-09 05:01:57 on dist-test-slave-21b9"
server_key: "aec649cd9ad5c4c0762a46c62ad63961"
server_key_iv: "b33bbe4b6664e4083de71026bf2665e6"
server_key_version: "encryptionkey@0"
I20260809 05:01:57.747347 1553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "2275149ec8dd45e5b917da07c76bc384"
format_stamp: "Formatted at 2026-08-09 05:01:57 on dist-test-slave-21b9"
server_key: "aec649cd9ad5c4c0762a46c62ad63961"
server_key_iv: "b33bbe4b6664e4083de71026bf2665e6"
server_key_version: "encryptionkey@0"
I20260809 05:01:57.752259 1553 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 05:01:57.755645 1578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:57.756868 1553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:01:57.757048 1553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "2275149ec8dd45e5b917da07c76bc384"
format_stamp: "Formatted at 2026-08-09 05:01:57 on dist-test-slave-21b9"
server_key: "aec649cd9ad5c4c0762a46c62ad63961"
server_key_iv: "b33bbe4b6664e4083de71026bf2665e6"
server_key_version: "encryptionkey@0"
I20260809 05:01:57.757241 1553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:57.775084 1553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:57.779017 1553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:57.779253 1553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:57.796305 1553 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:39523
I20260809 05:01:57.796330 1669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:39523 every 8 connection(s)
I20260809 05:01:57.797602 1553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:01:57.801353 1670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:57.805714 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 1553
I20260809 05:01:57.805922 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:01:57.806560 30671 external_mini_cluster.cc:1428] Setting key 84ec63e7b0ffeeea5c006cec00fc134b
I20260809 05:01:57.807950 1670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:01:57 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251717, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:57.829991 1673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:57.811046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35404 (local address 127.29.243.254:39523)
0809 05:01:57.812663 (+ 1617us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:57.812671 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:57.812726 (+ 55us) server_negotiation.cc:407] Connection header received
0809 05:01:57.814266 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:57.814273 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:57.814543 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:57.814741 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:57.815933 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:57.817592 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:57.818997 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:57.819764 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:57.819931 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:57.823404 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:57.823412 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:57.823423 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:57.823458 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:57.826052 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:57.826818 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:57.826824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:57.826826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:57.826983 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:57.827397 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:57.827400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:57.827402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:57.827862 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:57.828032 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:57.828696 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 05:01:57.828929 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1415,"spinlock_wait_cycles":169216,"thread_start_us":1236,"threads_started":1}
I20260809 05:01:57.830030 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:57.811054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:39523 (local address 127.0.0.1:35404)
0809 05:01:57.811991 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:57.812056 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:57.813905 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:57.814792 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:57.814797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:57.815384 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:57.815724 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:57.815741 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:57.817757 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:57.817761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:57.818851 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:57.818864 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:57.819986 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:57.819992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:57.820118 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:57.820726 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:57.820748 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:57.823161 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:57.826178 (+ 3017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:57.826190 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:57.826201 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:57.826699 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:57.827091 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:57.827096 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:57.827099 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:57.827307 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:57.828007 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:57.828017 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:57.828298 (+ 281us) client_negotiation.cc:769] Sending connection context
0809 05:01:57.828553 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 05:01:57.829123 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":97,"thread_start_us":410,"threads_started":1}
I20260809 05:01:57.832700 1670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:01:57.833652 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:57.833294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54150)
0809 05:01:57.833440 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:57.833547 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 05:01:57.833904 1585 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
I20260809 05:01:57.835357 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:57.835004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54154)
0809 05:01:57.835130 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:57.835195 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 05:01:57.835546 1670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:35083: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.29.243.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab.
Entry for principal kudu/127.29.243.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab.
I20260809 05:01:57.876847 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.253:35083
--webserver_interface=127.29.243.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab
--principal=kudu/127.29.243.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:39523,127.29.243.253:35083,127.29.243.252:36705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:01:57.884879 1670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } attempt: 1
I20260809 05:01:57.886186 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:57.885652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54170)
0809 05:01:57.885876 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:57.886025 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 05:01:57.887229 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:57.886879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54174)
0809 05:01:57.887008 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:57.887083 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 05:01:57.887518 1670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:35083: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
I20260809 05:01:57.955744 1670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } attempt: 2
I20260809 05:01:57.956943 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:57.956432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54186)
0809 05:01:57.956699 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:57.956793 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 05:01:57.958011 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:57.957667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54200)
0809 05:01:57.957809 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:57.957877 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 05:01:57.958211 1670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:35083: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
I20260809 05:01:58.049458 1670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } attempt: 3
I20260809 05:01:58.050611 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.050129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54208)
0809 05:01:58.050368 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.050464 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 05:01:58.051672 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.051318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54222)
0809 05:01:58.051465 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.051533 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 05:01:58.051893 1670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:35083: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
I20260809 05:01:58.195142 1670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } attempt: 4
I20260809 05:01:58.196537 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.195942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54232)
0809 05:01:58.196277 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.196366 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260809 05:01:58.197562 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.197188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54244)
0809 05:01:58.197330 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.197437 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 05:01:58.197762 1670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:35083: Network error: Client connection negotiation failed: client connection to 127.29.243.253:35083: connect: Connection refused (error 111)
W20260809 05:01:58.231385 1678 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:01:58.231700 1678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:58.231808 1678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:58.231930 1678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:58.239117 1678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:58.239220 1678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:58.239319 1678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:58.239395 1678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:58.239482 1678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:58.249185 1678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:39523,127.29.243.253:35083,127.29.243.252:36705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab
--principal=kudu/127.29.243.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.253:35083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.243.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:58.250526 1678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:58.251852 1678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:58.263332 1684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:58.263391 1685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:58.263733 1688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:58.264397 1678 server_base.cc:1034] running on GCE node
I20260809 05:01:58.265137 1678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:58.266532 1678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:58.267747 1678 hybrid_clock.cc:648] HybridClock initialized: now 1786251718267680 us; error 38 us; skew 500 ppm
I20260809 05:01:58.268056 1678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251718, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:58.271086 1678 init.cc:377] Logged in from keytab as kudu/127.29.243.253@KRBTEST.COM (short username kudu)
I20260809 05:01:58.272559 1678 webserver.cc:460] Webserver started at http://127.29.243.253:44637/ using document root <none> and password file <none>
I20260809 05:01:58.273258 1678 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:58.273355 1678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:58.273628 1678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:58.276583 1678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/data/instance:
uuid: "d61306fce30d47c490119e523ef02fa9"
format_stamp: "Formatted at 2026-08-09 05:01:58 on dist-test-slave-21b9"
server_key: "ab8ee0dd0632bdca7600063b217b14c0"
server_key_iv: "ed4062cce12e23762596827d312a4a37"
server_key_version: "encryptionkey@0"
I20260809 05:01:58.277324 1678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/wal/instance:
uuid: "d61306fce30d47c490119e523ef02fa9"
format_stamp: "Formatted at 2026-08-09 05:01:58 on dist-test-slave-21b9"
server_key: "ab8ee0dd0632bdca7600063b217b14c0"
server_key_iv: "ed4062cce12e23762596827d312a4a37"
server_key_version: "encryptionkey@0"
I20260809 05:01:58.282500 1678 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:01:58.286309 1701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:58.287690 1678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:01:58.287837 1678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/wal
uuid: "d61306fce30d47c490119e523ef02fa9"
format_stamp: "Formatted at 2026-08-09 05:01:58 on dist-test-slave-21b9"
server_key: "ab8ee0dd0632bdca7600063b217b14c0"
server_key_iv: "ed4062cce12e23762596827d312a4a37"
server_key_version: "encryptionkey@0"
I20260809 05:01:58.288024 1678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:58.311841 1678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:58.316236 1678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:58.316491 1678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:58.334481 1678 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.253:35083
I20260809 05:01:58.334494 1793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.253:35083 every 8 connection(s)
I20260809 05:01:58.335831 1678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
I20260809 05:01:58.339634 1795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:58.344218 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 1678
I20260809 05:01:58.344413 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-1/wal/instance
I20260809 05:01:58.344951 30671 external_mini_cluster.cc:1428] Setting key 81a4caf72c1897e05c2a2c110b513eea
I20260809 05:01:58.347093 1795 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251718, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:58.367815 1799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.350180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35416 (local address 127.29.243.254:39523)
0809 05:01:58.350661 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.350667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:58.351719 (+ 1052us) server_negotiation.cc:407] Connection header received
0809 05:01:58.353566 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:58.353572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:58.353655 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:58.353773 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:58.355257 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.356219 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.357655 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.358200 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.358315 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.361949 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:58.361954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:58.361957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:58.362005 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:58.364668 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.365378 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.365383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.365385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.365510 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.366041 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.366045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.366047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.366337 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:58.366458 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:58.367361 (+ 903us) server_negotiation.cc:299] Negotiation successful
0809 05:01:58.367538 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":229,"threads_started":1}
I20260809 05:01:58.369320 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.350730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:39523 (local address 127.0.0.1:35416)
0809 05:01:58.351547 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.351592 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:58.353303 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:58.353968 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:58.353975 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:58.354479 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:58.355042 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.355059 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.356353 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.356357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.357513 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.357525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.358399 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.358404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.358597 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.359212 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:58.359239 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:58.361722 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:58.364811 (+ 3089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.364821 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.364838 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.365259 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.365693 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.365698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.365702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.365929 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.366483 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:58.366504 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:58.366937 (+ 433us) client_negotiation.cc:769] Sending connection context
0809 05:01:58.367238 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 05:01:58.367789 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":321,"threads_started":1}
I20260809 05:01:58.371165 1795 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.243.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.252@KRBTEST.COM" created.
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251718, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
I20260809 05:01:58.389109 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.371840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54246)
0809 05:01:58.372044 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.372064 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:58.372178 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:58.373388 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:58.373392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:58.373765 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:58.373900 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.373908 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.375835 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.375839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.376689 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.376701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.377836 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.377841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.377966 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.378502 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:58.378519 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:58.383125 (+ 4606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:58.386886 (+ 3761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.386890 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.386893 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.387242 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.387636 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.387639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.387641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.387780 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.388415 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:58.388418 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:58.388539 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 05:01:58.388643 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 05:01:58.388832 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 05:01:58.389447 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.372187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54246 (local address 127.29.243.253:35083)
0809 05:01:58.372906 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.372913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:58.372944 (+ 31us) server_negotiation.cc:407] Connection header received
0809 05:01:58.372993 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:58.373001 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:58.373220 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:58.373355 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:58.374093 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.375656 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.376836 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.377681 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.377811 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.383422 (+ 5611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:58.383436 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:58.383452 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:58.383497 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:58.386761 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.387392 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.387400 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.387403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.387542 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.387878 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.387882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.387885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.388297 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:58.388516 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:58.388806 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 05:01:58.389046 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":393,"threads_started":1}
I20260809 05:01:58.391775 1795 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:01:58.392980 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.392483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50510)
0809 05:01:58.392658 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.392808 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 05:01:58.393275 1707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
I20260809 05:01:58.394767 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.394465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50516)
0809 05:01:58.394598 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.394673 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 05:01:58.394977 1795 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:36705: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab.
Entry for principal kudu/127.29.243.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab.
I20260809 05:01:58.418674 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.252:36705
--webserver_interface=127.29.243.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab
--principal=kudu/127.29.243.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:39523,127.29.243.253:35083,127.29.243.252:36705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:01:58.444304 1795 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } attempt: 1
I20260809 05:01:58.445706 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.445159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50520)
0809 05:01:58.445396 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.445540 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 05:01:58.446740 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.446408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50534)
0809 05:01:58.446550 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.446628 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 05:01:58.446943 1795 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:36705: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
I20260809 05:01:58.497007 1670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } attempt: 5
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251717, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
I20260809 05:01:58.511512 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.497859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54262 (local address 127.29.243.253:35083)
0809 05:01:58.498051 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.498055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:58.498080 (+ 25us) server_negotiation.cc:407] Connection header received
0809 05:01:58.498313 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:58.498317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:58.498436 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:58.498523 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:58.499593 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.500287 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.501945 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.502394 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.502490 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.506278 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:58.506283 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:58.506287 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:58.506320 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:58.508912 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.509715 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.509719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.509722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.509855 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.510249 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.510253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.510255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.510546 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:58.510663 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:58.511057 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 05:01:58.511226 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 05:01:58.511672 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.497751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54262)
0809 05:01:58.497997 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.498022 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:58.498161 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:58.498585 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:58.498590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:58.499110 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:58.499420 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.499436 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.500430 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.500435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.501772 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.501789 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.502556 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.502561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.502795 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.503513 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:58.503536 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:58.506073 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:58.509050 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.509055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.509059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.509566 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.509969 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.509974 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.509977 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.510154 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.510668 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:58.510673 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:58.510834 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 05:01:58.510987 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 05:01:58.511323 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 05:01:58.512814 1670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:01:58.513995 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.513573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50546)
0809 05:01:58.513746 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.513837 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 05:01:58.514964 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.514606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50562)
0809 05:01:58.514755 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.514835 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 05:01:58.515185 1670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:36705: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
I20260809 05:01:58.515151 1795 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } attempt: 2
I20260809 05:01:58.516005 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.515643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50572)
0809 05:01:58.515792 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.515866 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 05:01:58.516846 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.516536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50588)
0809 05:01:58.516667 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.516729 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 05:01:58.517030 1795 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:36705: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
I20260809 05:01:58.566469 1670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } attempt: 1
I20260809 05:01:58.567787 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.567188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50602)
0809 05:01:58.567448 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.567569 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260809 05:01:58.569037 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.568615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50614)
0809 05:01:58.568778 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.568891 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 05:01:58.569293 1670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:36705: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
I20260809 05:01:58.608246 1795 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } attempt: 3
I20260809 05:01:58.609562 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.608913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50618)
0809 05:01:58.609194 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.609332 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 05:01:58.610792 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.610246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50632)
0809 05:01:58.610523 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.610630 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260809 05:01:58.611006 1795 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:36705: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
I20260809 05:01:58.637522 1670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } attempt: 2
I20260809 05:01:58.638837 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.638241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50638)
0809 05:01:58.638507 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.638623 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 05:01:58.639883 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.639514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50642)
0809 05:01:58.639666 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.639754 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 05:01:58.640112 1670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:36705: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
I20260809 05:01:58.746349 1670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } attempt: 3
I20260809 05:01:58.747684 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.747049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50652)
0809 05:01:58.747326 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.747453 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 05:01:58.748889 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.748439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50660)
0809 05:01:58.748648 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.748753 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260809 05:01:58.749140 1670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:36705: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
I20260809 05:01:58.754240 1795 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } attempt: 4
I20260809 05:01:58.755324 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.754777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50666)
0809 05:01:58.755020 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.755121 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 05:01:58.756294 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.755948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50680)
0809 05:01:58.756087 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.756168 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 05:01:58.756536 1795 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:36705: Network error: Client connection negotiation failed: client connection to 127.29.243.252:36705: connect: Connection refused (error 111)
W20260809 05:01:58.772883 1809 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:01:58.773200 1809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:01:58.773270 1809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:01:58.773350 1809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:01:58.780468 1809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:01:58.780560 1809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:01:58.780622 1809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:01:58.780664 1809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:01:58.780716 1809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:01:58.790486 1809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:39523,127.29.243.253:35083,127.29.243.252:36705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab
--principal=kudu/127.29.243.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.252:36705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.243.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:01:58.791793 1809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:01:58.793164 1809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:01:58.804531 1815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:58.804633 1814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:01:58.805220 1819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:01:58.805694 1809 server_base.cc:1034] running on GCE node
I20260809 05:01:58.806489 1809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:01:58.807881 1809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:01:58.809120 1809 hybrid_clock.cc:648] HybridClock initialized: now 1786251718809044 us; error 38 us; skew 500 ppm
I20260809 05:01:58.809485 1809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251718, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:01:58.812794 1809 init.cc:377] Logged in from keytab as kudu/127.29.243.252@KRBTEST.COM (short username kudu)
I20260809 05:01:58.814263 1809 webserver.cc:460] Webserver started at http://127.29.243.252:42681/ using document root <none> and password file <none>
I20260809 05:01:58.814935 1809 fs_manager.cc:359] Metadata directory not provided
I20260809 05:01:58.815033 1809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:01:58.815399 1809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:01:58.818311 1809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/data/instance:
uuid: "1c224bae09c74ee985f2bacf76542031"
format_stamp: "Formatted at 2026-08-09 05:01:58 on dist-test-slave-21b9"
server_key: "361814a2a27a9c23d88d8d4bdd9adc70"
server_key_iv: "810d109d325a9b0e104db66a00670bb2"
server_key_version: "encryptionkey@0"
I20260809 05:01:58.819104 1809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/wal/instance:
uuid: "1c224bae09c74ee985f2bacf76542031"
format_stamp: "Formatted at 2026-08-09 05:01:58 on dist-test-slave-21b9"
server_key: "361814a2a27a9c23d88d8d4bdd9adc70"
server_key_iv: "810d109d325a9b0e104db66a00670bb2"
server_key_version: "encryptionkey@0"
I20260809 05:01:58.824065 1809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 05:01:58.827487 1833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:01:58.828760 1809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:01:58.828904 1809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/wal
uuid: "1c224bae09c74ee985f2bacf76542031"
format_stamp: "Formatted at 2026-08-09 05:01:58 on dist-test-slave-21b9"
server_key: "361814a2a27a9c23d88d8d4bdd9adc70"
server_key_iv: "810d109d325a9b0e104db66a00670bb2"
server_key_version: "encryptionkey@0"
I20260809 05:01:58.829128 1809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:01:58.842525 1809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:01:58.846640 1809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:01:58.846907 1809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:01:58.864176 1809 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.252:36705
I20260809 05:01:58.864197 1922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.252:36705 every 8 connection(s)
I20260809 05:01:58.865480 1809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
I20260809 05:01:58.869493 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 1809
I20260809 05:01:58.869666 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786251347713215-30671-0/minicluster-data/master-2/wal/instance
I20260809 05:01:58.869844 1925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:01:58.870122 30671 external_mini_cluster.cc:1428] Setting key 1c323e888850b609f2a7a761f7b0f65a
I20260809 05:01:58.879673 1925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251718, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:01:58.894137 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.871987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.29.243.252:36705)
0809 05:01:58.872764 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.872776 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:58.872819 (+ 43us) server_negotiation.cc:407] Connection header received
0809 05:01:58.874618 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:58.874643 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:58.875039 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:58.875402 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:58.877217 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.878489 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.880301 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.881046 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.881186 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.885723 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:58.885733 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:58.885748 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:58.885796 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:58.889401 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.890090 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.890097 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.890103 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.890281 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.890672 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.890675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.890677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.891382 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:58.891695 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:58.892148 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 05:01:58.892760 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":334,"threads_started":1}
I20260809 05:01:58.898762 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.882663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:39523 (local address 127.0.0.1:35430)
0809 05:01:58.883631 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.883684 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:58.883837 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:58.884260 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:58.884267 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:58.885064 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:58.885429 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.885442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.886785 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.886790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.888123 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.888137 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.889156 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.889161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.889312 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.889960 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:58.889979 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:58.892492 (+ 2513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:58.895991 (+ 3499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.896002 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.896009 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.896567 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.896965 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.896970 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.896974 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.897106 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.897695 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:58.897703 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:58.897886 (+ 183us) client_negotiation.cc:769] Sending connection context
0809 05:01:58.898044 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 05:01:58.898351 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"spinlock_wait_cycles":15616,"thread_start_us":543,"threads_started":1}
I20260809 05:01:58.898775 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.882718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35430 (local address 127.29.243.254:39523)
0809 05:01:58.883231 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.883238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:58.883778 (+ 540us) server_negotiation.cc:407] Connection header received
0809 05:01:58.883966 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:58.883972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:58.884059 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:58.884166 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:58.885599 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.886599 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.888283 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.888895 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.889048 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.892695 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:58.892703 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:58.892707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:58.892787 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:58.895866 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.896696 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.896702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.896705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.896859 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.897233 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.897238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.897241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.897565 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:58.897725 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:58.898190 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 05:01:58.898395 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":248,"threads_started":1}
I20260809 05:01:58.900753 1925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251718, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
I20260809 05:01:58.912755 1932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.897330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35444 (local address 127.29.243.254:39523)
0809 05:01:58.898133 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.898137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:58.898158 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:01:58.898197 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:58.898201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:58.898251 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:58.898332 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:58.899805 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.900920 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.902233 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.902829 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.902922 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.907158 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:58.907165 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:58.907170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:58.907211 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:58.909800 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.910767 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.910772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.910774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.910899 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.911310 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.911314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.911318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.911638 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:58.911903 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:58.912130 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 05:01:58.912315 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":540,"threads_started":1}
I20260809 05:01:58.914273 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.901373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:35083 (local address 127.0.0.1:54274)
0809 05:01:58.901535 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.901550 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:58.901628 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:58.902016 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:58.902021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:58.902436 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:58.902612 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.902624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.903871 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.903875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.904738 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.904749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.905653 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.905658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.905842 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.906325 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:58.906342 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:58.908393 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:58.911574 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.911579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.911582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.912059 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.912644 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.912649 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.912652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.912836 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.913408 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:58.913413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:58.913567 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 05:01:58.913692 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 05:01:58.913955 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 05:01:58.914383 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.901557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.29.243.253:35083)
0809 05:01:58.901747 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.901752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:58.901767 (+ 15us) server_negotiation.cc:407] Connection header received
0809 05:01:58.901813 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:58.901817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:58.901883 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:58.901963 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:58.902770 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.903729 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.904880 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.905502 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.905644 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.908580 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:58.908586 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:58.908591 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:58.908633 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:58.911454 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.912180 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.912185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.912188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.912345 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.913023 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.913027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.913029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.913280 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:58.913421 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:58.913800 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 05:01:58.914006 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:01:58.915563 1925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251718, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
I20260809 05:01:58.926344 1670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } attempt: 4
I20260809 05:01:58.929600 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.916239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50696)
0809 05:01:58.916487 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.916501 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:58.916580 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:58.916832 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:58.916835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:58.917168 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:58.917308 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.917318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.918219 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.918225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.919295 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.919310 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.920098 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.920104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.920306 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.920785 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:58.920808 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:58.923214 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:58.926778 (+ 3564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.926783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.926786 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.927243 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.927697 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.927701 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.927705 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.927865 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.928509 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:58.928514 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:58.928667 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 05:01:58.928815 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 05:01:58.929175 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260809 05:01:58.929783 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.916229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50696 (local address 127.29.243.252:36705)
0809 05:01:58.916372 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.916377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:58.916555 (+ 178us) server_negotiation.cc:407] Connection header received
0809 05:01:58.916650 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:58.916655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:58.916722 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:58.916808 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:58.917443 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.918087 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.919421 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.919933 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.920028 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.923390 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:58.923396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:58.923401 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:58.923432 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:58.926638 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.927413 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.927419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.927422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.927567 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.928003 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.928009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.928012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.928392 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:58.928541 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:58.928942 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 05:01:58.929382 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":143,"server-negotiator.queue_time_us":40}
I20260809 05:01:58.930892 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.914628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.29.243.253:35083)
0809 05:01:58.914798 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.914802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:58.914816 (+ 14us) server_negotiation.cc:407] Connection header received
0809 05:01:58.914852 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:58.914855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:58.914914 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:58.914988 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:58.916372 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.917245 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.918571 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.918966 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.919061 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.924356 (+ 5295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:58.924361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:58.924366 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:58.924405 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:58.927537 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.928363 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.928368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.928371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.928526 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.928903 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.928908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.928912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.929238 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:58.929491 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:58.929986 (+ 495us) server_negotiation.cc:299] Negotiation successful
0809 05:01:58.930220 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":82}
Aug 09 05:01:58 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251717, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
I20260809 05:01:58.936403 1925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031: Bootstrap starting.
I20260809 05:01:58.936741 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:01:58.940743 1925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:58.942086 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.926919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50706)
0809 05:01:58.927202 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:58.927223 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:58.927364 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:58.928148 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:58.928153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:58.928603 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:58.928829 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.928843 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.930868 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.930872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.932033 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:58.932046 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.932882 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.932888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:58.933099 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.933666 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:58.933683 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:58.936202 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:58.939264 (+ 3062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.939296 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.939299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.939765 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.940191 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:58.940196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:58.940199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:58.940379 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:58.940998 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:58.941003 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:58.941216 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 05:01:58.941366 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 05:01:58.941665 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260809 05:01:58.942147 1933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.926980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.29.243.252:36705)
0809 05:01:58.927868 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.927872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:58.927892 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:01:58.927929 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:58.927932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:58.927990 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:58.928088 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:58.929011 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.930744 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.932157 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:58.932733 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:58.932867 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:58.936446 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:58.936453 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:58.936457 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:58.936494 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:58.939138 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.939904 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.939909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.939912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.940068 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:58.940509 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:58.940515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:58.940527 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:58.940876 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:58.941028 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:58.941494 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 05:01:58.941713 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":792,"thread_start_us":577,"threads_started":1}
I20260809 05:01:58.942155 1925 log.cc:824] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:58.940141 1621 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2275149ec8dd45e5b917da07c76bc384"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35444
I20260809 05:01:58.946833 1670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384: Bootstrap starting.
I20260809 05:01:58.948642 1925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031: No bootstrap required, opened a new log
I20260809 05:01:58.950311 1670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:58.951288 1670 log.cc:824] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:58.953797 1670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384: No bootstrap required, opened a new log
I20260809 05:01:58.958668 1670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } }
I20260809 05:01:58.958979 1670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2275149ec8dd45e5b917da07c76bc384, State: Initialized, Role: FOLLOWER
I20260809 05:01:58.959141 1925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } }
I20260809 05:01:58.959506 1925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c224bae09c74ee985f2bacf76542031, State: Initialized, Role: FOLLOWER
I20260809 05:01:58.959481 1670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } }
I20260809 05:01:58.959971 1925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } }
I20260809 05:01:58.960464 1944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } } }
I20260809 05:01:58.960884 1944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:01:58.961330 1945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } } }
I20260809 05:01:58.961747 1945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:01:58.961848 1670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:58.961920 1621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 05:01:58.962095 1621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:58.962168 1925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:58.965550 1621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } }
I20260809 05:01:58.968336 1881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2275149ec8dd45e5b917da07c76bc384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1c224bae09c74ee985f2bacf76542031"
I20260809 05:01:58.968855 1881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:58.970093 1621 leader_election.cc:288] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [CANDIDATE]: Term 1 election: Requested vote from peers d61306fce30d47c490119e523ef02fa9 (127.29.243.253:35083), 1c224bae09c74ee985f2bacf76542031 (127.29.243.252:36705)
I20260809 05:01:58.972611 1881 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2275149ec8dd45e5b917da07c76bc384 in term 1.
I20260809 05:01:58.973281 1582 leader_election.cc:302] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c224bae09c74ee985f2bacf76542031, 2275149ec8dd45e5b917da07c76bc384; no voters:
I20260809 05:01:58.973811 1944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:01:58.980405 1944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [term 1 LEADER]: Becoming Leader. State: Replica: 2275149ec8dd45e5b917da07c76bc384, State: Running, Role: LEADER
I20260809 05:01:58.981050 1944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } }
I20260809 05:01:58.990873 1964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2275149ec8dd45e5b917da07c76bc384. Latest consensus state: current_term: 1 leader_uuid: "2275149ec8dd45e5b917da07c76bc384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } } }
I20260809 05:01:58.991298 1964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:58.992652 1977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:01:59.007869 1977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 05:01:59.032692 1587 tablet.cc:2369] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:01:59.029831 1553 master_runner.cc:186] Error getting master registration for 127.29.243.253:35083: OK, instance_id { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" instance_seqno: 1786251718329489 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 05:01:59.051537 1553 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 05:01:59.056347 1933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.993541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50712 (local address 127.29.243.252:36705)
0809 05:01:58.993738 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.993745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.006994 (+ 13249us) server_negotiation.cc:407] Connection header received
0809 05:01:59.009926 (+ 2932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.009976 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.010239 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.010382 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.027698 (+ 17316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:59.028768 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:59.031984 (+ 3216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:59.032646 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:59.032757 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:59.044041 (+ 11284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:59.044049 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:59.044055 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:59.044105 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:59.050395 (+ 6290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:59.051864 (+ 1469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:59.051873 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:59.051877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:59.052173 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:59.053011 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:59.053028 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:59.053032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:59.053704 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:59.054101 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:59.055332 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0809 05:01:59.055779 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 05:01:59.056996 1795 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } attempt: 5
I20260809 05:01:59.058034 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:59.014088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54300 (local address 127.29.243.253:35083)
0809 05:01:59.014258 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.014263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.019292 (+ 5029us) server_negotiation.cc:407] Connection header received
0809 05:01:59.020355 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.020362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.020460 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.020540 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.024405 (+ 3865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:59.025520 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:59.034434 (+ 8914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:59.035101 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:59.035232 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:59.049998 (+ 14766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:59.050006 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:59.050012 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:59.050064 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:59.053814 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:59.055588 (+ 1774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:59.055595 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:59.055599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:59.055771 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:59.056215 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:59.056221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:59.056225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:59.056600 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:59.056834 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:59.057417 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 05:01:59.057667 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 05:01:59.063090 1932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:58.994699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35448 (local address 127.29.243.254:39523)
0809 05:01:58.994911 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:58.994918 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.013457 (+ 18539us) server_negotiation.cc:407] Connection header received
0809 05:01:59.015131 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.015137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.015220 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.015380 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.029616 (+ 14236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:59.030750 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:59.032667 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:59.033313 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:59.033409 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:59.044792 (+ 11383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:59.044800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:59.044805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:59.044852 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:59.048929 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:59.054050 (+ 5121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:59.054057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:59.054061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:59.054228 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:59.055431 (+ 1203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:59.055437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:59.055441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:59.061802 (+ 6361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:59.062073 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:59.062435 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 05:01:59.062698 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 05:01:59 dist-test-slave-21b9 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251718, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
I20260809 05:01:59.078207 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:59.057671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:36705 (local address 127.0.0.1:50716)
0809 05:01:59.058044 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 05:01:59.058063 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:01:59.058168 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:01:59.058832 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:01:59.058837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:01:59.059623 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:01:59.059841 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:59.059854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:59.061355 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:59.061360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:59.062723 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:01:59.062739 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:01:59.064043 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:01:59.064048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:01:59.064260 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:59.064853 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:01:59.064888 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:01:59.070307 (+ 5419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:01:59.074951 (+ 4644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:59.074956 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:59.074960 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:59.075458 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:59.075927 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:01:59.075932 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:01:59.075935 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:01:59.076156 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:01:59.077186 (+ 1030us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:01:59.077191 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:01:59.077355 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 05:01:59.077510 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 05:01:59.077821 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248}
I20260809 05:01:59.076857 1881 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [term 1 FOLLOWER]: Refusing update from remote peer 2275149ec8dd45e5b917da07c76bc384: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:59.083104 1933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:01:59.058358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50716 (local address 127.29.243.252:36705)
0809 05:01:59.058532 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.058537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.058554 (+ 17us) server_negotiation.cc:407] Connection header received
0809 05:01:59.058599 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.058603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.058667 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.059987 (+ 1320us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.060126 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:59.061230 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:59.063038 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:01:59.063869 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:01:59.064558 (+ 689us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:01:59.070561 (+ 6003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:01:59.070569 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:01:59.070574 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:01:59.070615 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:01:59.074786 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:59.075602 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:59.075607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:59.075611 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:59.075796 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:01:59.076331 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:01:59.076337 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:01:59.076364 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:01:59.077053 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:01:59.077237 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 05:01:59.082497 (+ 5260us) server_negotiation.cc:299] Negotiation successful
0809 05:01:59.082729 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 05:01:59.091629 1964 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 05:01:59.099001 1843 tablet.cc:2369] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:01:59.100768 1795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9: Bootstrap starting.
I20260809 05:01:59.104956 1795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9: Neither blocks nor log segments found. Creating new log.
I20260809 05:01:59.092108 1809 master_runner.cc:186] Error getting master registration for 127.29.243.253:35083: OK, instance_id { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" instance_seqno: 1786251718329489 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 05:01:59.106009 1809 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 05:01:59.106586 1795 log.cc:824] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9: Log is configured to *not* fsync() on all Append() calls
I20260809 05:01:59.113674 1795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9: No bootstrap required, opened a new log
I20260809 05:01:59.120386 1945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2275149ec8dd45e5b917da07c76bc384. Latest consensus state: current_term: 1 leader_uuid: "2275149ec8dd45e5b917da07c76bc384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } } }
I20260809 05:01:59.120740 1945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:01:59.124049 2012 mvcc.cc:205] Tried to move back new op lower bound from 7316487041302675456 to 7316487040952107008. Current Snapshot: MvccSnapshot[applied={T|T < 7316487041302675456}]
W20260809 05:01:59.129693 2010 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:01:59.129797 2010 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:01:59.130498 1945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2275149ec8dd45e5b917da07c76bc384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } } }
I20260809 05:01:59.130760 1945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:01:59.136476 1795 raft_consensus.cc:348] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } }
I20260809 05:01:59.136960 1795 raft_consensus.cc:729] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61306fce30d47c490119e523ef02fa9, State: Initialized, Role: FOLLOWER
I20260809 05:01:59.137569 1795 consensus_queue.cc:260] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } }
I20260809 05:01:59.142499 1964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2275149ec8dd45e5b917da07c76bc384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } } }
I20260809 05:01:59.142966 1964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:59.147210 1977 catalog_manager.cc:1347] Generated new cluster ID: 57c858f051b44ee98fb9dea652946c63
I20260809 05:01:59.147374 1977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:01:59.169524 2019 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } } }
I20260809 05:01:59.171646 1753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2275149ec8dd45e5b917da07c76bc384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d61306fce30d47c490119e523ef02fa9"
I20260809 05:01:59.172042 1752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:01:59.171466 2019 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:01:59.176748 1752 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [term 1 FOLLOWER]: Refusing update from remote peer 2275149ec8dd45e5b917da07c76bc384: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:01:59.177448 1795 sys_catalog.cc:564] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:01:59.179395 1964 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:01:59.183209 1753 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2275149ec8dd45e5b917da07c76bc384 in term 1.
I20260809 05:01:59.192901 1977 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:01:59.194733 1977 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:01:59.216271 1977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384: Generated new TSK 0
I20260809 05:01:59.217358 1977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:01:59.221691 2019 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2275149ec8dd45e5b917da07c76bc384. Latest consensus state: current_term: 1 leader_uuid: "2275149ec8dd45e5b917da07c76bc384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } } }
I20260809 05:01:59.222012 2019 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:01:59.222824 2026 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2275149ec8dd45e5b917da07c76bc384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } } }
I20260809 05:01:59.223248 2026 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:01:59.223174 1944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2275149ec8dd45e5b917da07c76bc384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2275149ec8dd45e5b917da07c76bc384" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 39523 } } peers { permanent_uuid: "d61306fce30d47c490119e523ef02fa9" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 35083 } } peers { permanent_uuid: "1c224bae09c74ee985f2bacf76542031" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 36705 } } }
I20260809 05:01:59.223533 1944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2275149ec8dd45e5b917da07c76bc384 [sys.catalog]: This master's current role is: LEADER
I20260809 05:01:59.249703 2046 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 05:01:59.330024 1932 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.323747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35450 (local address 127.29.243.254:39523)
0809 05:01:59.323940 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.323946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.328137 (+ 4191us) server_negotiation.cc:407] Connection header received
0809 05:01:59.328376 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.328381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.328456 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.328522 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.329738 (+ 1216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35450: BlockingRecv error: recv got EOF from 127.0.0.1:35450 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 05:01:59.332424 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.328278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54310 (local address 127.29.243.253:35083)
0809 05:01:59.328453 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.328459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.330875 (+ 2416us) server_negotiation.cc:407] Connection header received
0809 05:01:59.331128 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.331134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.331219 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.331322 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.332193 (+ 871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54310: BlockingRecv error: recv got EOF from 127.0.0.1:54310 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 05:01:59.333675 1933 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.320436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50722 (local address 127.29.243.252:36705)
0809 05:01:59.320633 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.320639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.327856 (+ 7217us) server_negotiation.cc:407] Connection header received
0809 05:01:59.328190 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.328196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.328304 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.328380 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.333436 (+ 5056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50722: BlockingRecv error: recv got EOF from 127.0.0.1:50722 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 05:01:59.367991 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.365583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54312 (local address 127.29.243.253:35083)
0809 05:01:59.365772 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.365778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.366313 (+ 535us) server_negotiation.cc:407] Connection header received
0809 05:01:59.366559 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.366565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.366694 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.366809 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.367714 (+ 905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54312: BlockingRecv error: recv got EOF from 127.0.0.1:54312 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 05:01:59.379621 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.377433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54314 (local address 127.29.243.253:35083)
0809 05:01:59.377607 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.377613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.378167 (+ 554us) server_negotiation.cc:407] Connection header received
0809 05:01:59.378342 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.378348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.378446 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.378553 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.379402 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54314: BlockingRecv error: recv got EOF from 127.0.0.1:54314 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 05:01:59.389241 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.384591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54322 (local address 127.29.243.253:35083)
0809 05:01:59.384746 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.384752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.387811 (+ 3059us) server_negotiation.cc:407] Connection header received
0809 05:01:59.387994 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.388000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.388094 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.388205 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.388993 (+ 788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54322: BlockingRecv error: recv got EOF from 127.0.0.1:54322 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 05:01:59.401857 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.399358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.29.243.253:35083)
0809 05:01:59.399601 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.399607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.400292 (+ 685us) server_negotiation.cc:407] Connection header received
0809 05:01:59.400632 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.400638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.400735 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.400856 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.401625 (+ 769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54338: BlockingRecv error: recv got EOF from 127.0.0.1:54338 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
W20260809 05:01:59.415361 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.413101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54354 (local address 127.29.243.253:35083)
0809 05:01:59.413283 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.413289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.413845 (+ 556us) server_negotiation.cc:407] Connection header received
0809 05:01:59.414004 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.414010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.414114 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.414224 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.415062 (+ 838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54354: BlockingRecv error: recv got EOF from 127.0.0.1:54354 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 05:01:59.437156 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.434751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54366 (local address 127.29.243.253:35083)
0809 05:01:59.434936 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.434942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.435523 (+ 581us) server_negotiation.cc:407] Connection header received
0809 05:01:59.435736 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.435742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.435860 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.435994 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.436890 (+ 896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54366: BlockingRecv error: recv got EOF from 127.0.0.1:54366 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 05:01:59.478379 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.475933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.29.243.253:35083)
0809 05:01:59.476147 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.476153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.476704 (+ 551us) server_negotiation.cc:407] Connection header received
0809 05:01:59.476909 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.476915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.477013 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.477119 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.478083 (+ 964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54376: BlockingRecv error: recv got EOF from 127.0.0.1:54376 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260809 05:01:59.547947 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.545910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.29.243.253:35083)
0809 05:01:59.546121 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.546127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.546532 (+ 405us) server_negotiation.cc:407] Connection header received
0809 05:01:59.546720 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.546726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.546821 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.546947 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.547721 (+ 774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54386: BlockingRecv error: recv got EOF from 127.0.0.1:54386 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 05:01:59.681926 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.679886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.29.243.253:35083)
0809 05:01:59.680138 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.680145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.680412 (+ 267us) server_negotiation.cc:407] Connection header received
0809 05:01:59.680648 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.680654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.680750 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.680842 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.681671 (+ 829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54394: BlockingRecv error: recv got EOF from 127.0.0.1:54394 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 05:01:59.944543 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:01:59.942735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54410 (local address 127.29.243.253:35083)
0809 05:01:59.942928 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 05:01:59.942933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:01:59.943172 (+ 239us) server_negotiation.cc:407] Connection header received
0809 05:01:59.943429 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:01:59.943433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:01:59.943513 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:01:59.943611 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:01:59.944330 (+ 719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54410: BlockingRecv error: recv got EOF from 127.0.0.1:54410 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 05:02:00.131783 2010 catalog_manager.cc:1259] Loaded cluster ID: 57c858f051b44ee98fb9dea652946c63
I20260809 05:02:00.131887 2010 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031: loading cluster ID for follower catalog manager: success
I20260809 05:02:00.134584 2010 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031: acquiring CA information for follower catalog manager: success
I20260809 05:02:00.136411 2010 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1c224bae09c74ee985f2bacf76542031: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 05:02:00.254619 2046 catalog_manager.cc:1259] Loaded cluster ID: 57c858f051b44ee98fb9dea652946c63
I20260809 05:02:00.254737 2046 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9: loading cluster ID for follower catalog manager: success
I20260809 05:02:00.257364 2046 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9: acquiring CA information for follower catalog manager: success
I20260809 05:02:00.259225 2046 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d61306fce30d47c490119e523ef02fa9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 05:02:00.469383 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:00.462693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54418 (local address 127.29.243.253:35083)
0809 05:02:00.463318 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:00.463325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:00.463680 (+ 355us) server_negotiation.cc:407] Connection header received
0809 05:02:00.463900 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:00.463905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:00.464066 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:00.464186 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:00.464688 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:00.465557 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:00.466896 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:00.467375 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:00.467493 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:00.468077 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:00.468688 (+ 611us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:00.468843 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:00.468924 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 05:02:00.469049 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":299,"threads_started":1}
I20260809 05:02:00.483687 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:00.475744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50728 (local address 127.29.243.252:36705)
0809 05:02:00.477717 (+ 1973us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:00.477724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:00.477744 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:02:00.477853 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:00.477858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:00.478079 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:00.478263 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:00.478764 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:00.479871 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:00.481173 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:00.481748 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:00.481883 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:00.482331 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:00.483038 (+ 707us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:00.483197 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:00.483250 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 05:02:00.483363 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1849,"thread_start_us":1751,"threads_started":1}
I20260809 05:02:00.510182 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:00.490600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50736 (local address 127.29.243.252:36705)
0809 05:02:00.490770 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:00.490775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:00.502995 (+ 12220us) server_negotiation.cc:407] Connection header received
0809 05:02:00.503441 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:00.503447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:00.503545 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:00.503609 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:00.504432 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:00.505543 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:00.507897 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:00.508511 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:00.508628 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:00.509335 (+ 707us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:00.509481 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:00.509547 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:00.509786 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 05:02:00.509874 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 05:02:00.511240 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:00.497742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54428 (local address 127.29.243.253:35083)
0809 05:02:00.497965 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:00.497971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:00.504334 (+ 6363us) server_negotiation.cc:407] Connection header received
0809 05:02:00.504519 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:00.504525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:00.504623 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:00.504714 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:00.505431 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:00.506547 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:00.508745 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:00.509496 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:00.509592 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:00.510189 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:00.510303 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:00.510389 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:00.510815 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 05:02:00.510892 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:02:00.511572 2185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:00.490385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.29.243.254:39523)
0809 05:02:00.503258 (+ 12873us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:00.503264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:00.503304 (+ 40us) server_negotiation.cc:407] Connection header received
0809 05:02:00.503429 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:00.503435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:00.503661 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:00.505033 (+ 1372us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:00.505205 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:00.506276 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:00.508513 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:00.509208 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:00.509789 (+ 581us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:00.509997 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:00.510562 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:00.511136 (+ 574us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:00.511189 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 05:02:00.511290 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12756,"thread_start_us":12637,"threads_started":1}
I20260809 05:02:00.517032 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1553
I20260809 05:02:00.518039 1662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:02:00.652519 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 1553
I20260809 05:02:00.680727 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1678
I20260809 05:02:00.686044 1787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:02:00.792361 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 1678
I20260809 05:02:00.822388 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 1809
I20260809 05:02:00.823604 1914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:02:00.929261 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 1809
2026-08-09T05:02:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5159 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:02:01 dist-test-slave-21b9 krb5kdc[2195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:02:01 dist-test-slave-21b9 krb5kdc[2195](info): set up 2 sockets
Aug 09 05:02:01 dist-test-slave-21b9 krb5kdc[2195](info): commencing operation
krb5kdc: starting...
W20260809 05:02:02.194794 30671 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/test-user.keytab.
Aug 09 05:02:02 dist-test-slave-21b9 krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:02:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:02:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.243.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
Entry for principal kudu/127.29.243.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab.
I20260809 05:02:02.518720 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41707
--webserver_interface=127.29.243.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41707,127.29.243.253:42549,127.29.243.252:44801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:02:02.856688 2211 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:02:02.856997 2211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:02:02.857095 2211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:02:02.857203 2211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:02:02.864272 2211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:02:02.864360 2211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:02:02.864428 2211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:02:02.864502 2211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:02:02.864588 2211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:02:02.874275 2211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41707,127.29.243.253:42549,127.29.243.252:44801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.254.keytab
--principal=kudu/127.29.243.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.254:41707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.243.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:02:02.875605 2211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:02:02.876924 2211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:02:02.888311 2224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:02:02.888633 2220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:02:02.888660 2221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:02:02.889386 2211 server_base.cc:1034] running on GCE node
I20260809 05:02:02.890126 2211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:02:02.891455 2211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:02:02.892668 2211 hybrid_clock.cc:648] HybridClock initialized: now 1786251722892606 us; error 32 us; skew 500 ppm
I20260809 05:02:02.892954 2211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:02:02 dist-test-slave-21b9 krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251722, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:02:02.895925 2211 init.cc:377] Logged in from keytab as kudu/127.29.243.254@KRBTEST.COM (short username kudu)
I20260809 05:02:02.897349 2211 webserver.cc:460] Webserver started at http://127.29.243.254:41503/ using document root <none> and password file <none>
I20260809 05:02:02.898047 2211 fs_manager.cc:359] Metadata directory not provided
I20260809 05:02:02.898152 2211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:02:02.898473 2211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:02:02.901432 2211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/data/instance:
uuid: "563e8f1dbe46485b8eb7158102ccdd8e"
format_stamp: "Formatted at 2026-08-09 05:02:02 on dist-test-slave-21b9"
server_key: "1cfb392800e11e0239eace0e783c4657"
server_key_iv: "92267c9223ba3e58656dcda12e6f373f"
server_key_version: "encryptionkey@0"
I20260809 05:02:02.902204 2211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/wal/instance:
uuid: "563e8f1dbe46485b8eb7158102ccdd8e"
format_stamp: "Formatted at 2026-08-09 05:02:02 on dist-test-slave-21b9"
server_key: "1cfb392800e11e0239eace0e783c4657"
server_key_iv: "92267c9223ba3e58656dcda12e6f373f"
server_key_version: "encryptionkey@0"
I20260809 05:02:02.907290 2211 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 05:02:02.910706 2233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:02:02.912093 2211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:02:02.912282 2211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/wal
uuid: "563e8f1dbe46485b8eb7158102ccdd8e"
format_stamp: "Formatted at 2026-08-09 05:02:02 on dist-test-slave-21b9"
server_key: "1cfb392800e11e0239eace0e783c4657"
server_key_iv: "92267c9223ba3e58656dcda12e6f373f"
server_key_version: "encryptionkey@0"
I20260809 05:02:02.912506 2211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:02:02.933459 2211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:02:02.937656 2211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:02:02.937958 2211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:02:02.957082 2211 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.254:41707
I20260809 05:02:02.957101 2320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.254:41707 every 8 connection(s)
I20260809 05:02:02.958544 2211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/data/info.pb
I20260809 05:02:02.960438 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 2211
I20260809 05:02:02.960665 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-0/wal/instance
I20260809 05:02:02.961263 30671 external_mini_cluster.cc:1428] Setting key 36d113022acb342813c0e42452166c7d
I20260809 05:02:02.963208 2322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:02:02.971845 2322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:02:02 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251722, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:02:02.996548 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:02.975659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41707 (local address 127.0.0.1:39544)
0809 05:02:02.976573 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:02.976631 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 05:02:02.978842 (+ 2211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:02:02.979915 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:02:02.979920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:02:02.980469 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:02:02.980804 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:02.980820 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:02.982945 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:02.982949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:02.984107 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:02.984121 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:02.985368 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:02.985375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:02.985527 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:02.986150 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:02:02.986174 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:02:02.988428 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:02:02.992424 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:02.992434 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:02.992445 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:02.992842 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:02.993355 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:02.993358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:02.993361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:02.993532 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:02.994534 (+ 1002us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:02:02.994540 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:02:02.994759 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 05:02:02.994966 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 05:02:02.995419 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":2,"thread_start_us":369,"threads_started":1}
I20260809 05:02:02.996585 2325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:02.975659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39544 (local address 127.29.243.254:41707)
0809 05:02:02.977493 (+ 1834us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:02.977502 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:02.977549 (+ 47us) server_negotiation.cc:407] Connection header received
0809 05:02:02.979255 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:02.979263 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:02.979654 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:02.979864 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:02.981020 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:02.982786 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:02.984251 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:02.985133 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:02.985307 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:02.988680 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:02.988691 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:02.988710 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:02.988744 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:02.992295 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:02.993008 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:02.993018 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:02.993022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:02.993240 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:02.993651 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:02.993656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:02.993659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:02.994397 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:02.994636 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:02.995156 (+ 520us) server_negotiation.cc:299] Negotiation successful
0809 05:02:02.995582 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":1632,"thread_start_us":1391,"threads_started":1}
WARNING: no policy specified for kudu/127.29.243.253@KRBTEST.COM; defaulting to no policy
I20260809 05:02:02.999651 2322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.253@KRBTEST.COM" created.
I20260809 05:02:03.000864 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.000365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46712)
0809 05:02:03.000618 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.000732 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260809 05:02:03.001151 2234 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
I20260809 05:02:03.002729 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.002371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46724)
0809 05:02:03.002515 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.002588 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 05:02:03.002933 2322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:42549: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab.
Entry for principal kudu/127.29.243.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab.
I20260809 05:02:03.038774 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.253:42549
--webserver_interface=127.29.243.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab
--principal=kudu/127.29.243.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41707,127.29.243.253:42549,127.29.243.252:44801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:02:03.052312 2322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } attempt: 1
I20260809 05:02:03.053802 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.053240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46728)
0809 05:02:03.053482 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.053635 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 05:02:03.054836 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.054400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46736)
0809 05:02:03.054571 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.054649 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 05:02:03.055076 2322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:42549: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
I20260809 05:02:03.123390 2322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } attempt: 2
I20260809 05:02:03.124559 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.124068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46752)
0809 05:02:03.124317 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.124411 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 05:02:03.125566 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.125210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46756)
0809 05:02:03.125366 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.125435 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 05:02:03.125758 2322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:42549: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
I20260809 05:02:03.216977 2322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } attempt: 3
I20260809 05:02:03.218231 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.217743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46766)
0809 05:02:03.217956 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.218088 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 05:02:03.219215 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.218841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46768)
0809 05:02:03.218986 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.219065 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 05:02:03.219488 2322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:42549: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
I20260809 05:02:03.362763 2322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } attempt: 4
I20260809 05:02:03.363979 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.363438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46772)
0809 05:02:03.363750 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.363837 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260809 05:02:03.364908 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.364572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46788)
0809 05:02:03.364710 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.364788 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 05:02:03.365096 2322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.253:42549: Network error: Client connection negotiation failed: client connection to 127.29.243.253:42549: connect: Connection refused (error 111)
W20260809 05:02:03.394145 2331 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:02:03.394488 2331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:02:03.394589 2331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:02:03.394704 2331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:02:03.402026 2331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:02:03.402128 2331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:02:03.402209 2331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:02:03.402277 2331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:02:03.402366 2331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:02:03.412485 2331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41707,127.29.243.253:42549,127.29.243.252:44801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.253.keytab
--principal=kudu/127.29.243.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.253:42549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.243.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:02:03.413827 2331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:02:03.415155 2331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:02:03.426970 2341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:02:03.426935 2339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:02:03.427356 2338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:02:03.428049 2331 server_base.cc:1034] running on GCE node
I20260809 05:02:03.428802 2331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:02:03.430336 2331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:02:03.431559 2331 hybrid_clock.cc:648] HybridClock initialized: now 1786251723431494 us; error 35 us; skew 500 ppm
I20260809 05:02:03.431896 2331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:02:03 dist-test-slave-21b9 krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:02:03.434979 2331 init.cc:377] Logged in from keytab as kudu/127.29.243.253@KRBTEST.COM (short username kudu)
I20260809 05:02:03.436561 2331 webserver.cc:460] Webserver started at http://127.29.243.253:36265/ using document root <none> and password file <none>
I20260809 05:02:03.437348 2331 fs_manager.cc:359] Metadata directory not provided
I20260809 05:02:03.437458 2331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:02:03.437772 2331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:02:03.440828 2331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/data/instance:
uuid: "d2c34298aed949d798eabf0c66bce14a"
format_stamp: "Formatted at 2026-08-09 05:02:03 on dist-test-slave-21b9"
server_key: "743b358baaa661563731d0acbd1e1386"
server_key_iv: "9c25418ae81c4d0128f8c270aaac1c0d"
server_key_version: "encryptionkey@0"
I20260809 05:02:03.441603 2331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/wal/instance:
uuid: "d2c34298aed949d798eabf0c66bce14a"
format_stamp: "Formatted at 2026-08-09 05:02:03 on dist-test-slave-21b9"
server_key: "743b358baaa661563731d0acbd1e1386"
server_key_iv: "9c25418ae81c4d0128f8c270aaac1c0d"
server_key_version: "encryptionkey@0"
I20260809 05:02:03.446820 2331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 05:02:03.450544 2355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:02:03.452019 2331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 05:02:03.452195 2331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/wal
uuid: "d2c34298aed949d798eabf0c66bce14a"
format_stamp: "Formatted at 2026-08-09 05:02:03 on dist-test-slave-21b9"
server_key: "743b358baaa661563731d0acbd1e1386"
server_key_iv: "9c25418ae81c4d0128f8c270aaac1c0d"
server_key_version: "encryptionkey@0"
I20260809 05:02:03.452400 2331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:02:03.475178 2331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:02:03.479604 2331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:02:03.479864 2331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:02:03.497716 2331 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.253:42549
I20260809 05:02:03.497758 2433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.253:42549 every 8 connection(s)
I20260809 05:02:03.499094 2331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/data/info.pb
I20260809 05:02:03.502781 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 2331
I20260809 05:02:03.502954 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-1/wal/instance
I20260809 05:02:03.502983 2434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:02:03.503437 30671 external_mini_cluster.cc:1428] Setting key 5e111fa1808c4b7c1d1bfa86973439ac
I20260809 05:02:03.510667 2434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:02:03 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:02:03.533658 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.514473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39560 (local address 127.29.243.254:41707)
0809 05:02:03.515001 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:03.515009 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:03.515504 (+ 495us) server_negotiation.cc:407] Connection header received
0809 05:02:03.517549 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:03.517555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:03.517661 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:03.517778 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:03.519771 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.520898 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.522455 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.523044 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.523165 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:03.526920 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:03.526927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:03.526931 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:03.526978 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:03.530288 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:03.531102 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:03.531107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:03.531110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:03.531256 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:03.531738 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:03.531743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:03.531745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:03.532112 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:03.532288 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:03.533045 (+ 757us) server_negotiation.cc:299] Negotiation successful
0809 05:02:03.533264 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":261,"threads_started":1}
I20260809 05:02:03.534552 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.514291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41707 (local address 127.0.0.1:39560)
0809 05:02:03.515337 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.515391 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 05:02:03.517255 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:02:03.518074 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:02:03.518086 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:02:03.518821 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:02:03.519549 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:03.519569 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.521068 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.521073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:03.522295 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:03.522309 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.523261 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.523267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:03.523524 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:03.524175 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:02:03.524198 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:02:03.526648 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:02:03.530470 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:03.530478 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:03.530511 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:03.530956 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:03.531393 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:03.531397 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:03.531399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:03.531615 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:03.532280 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:02:03.532287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:02:03.532666 (+ 379us) client_negotiation.cc:769] Sending connection context
0809 05:02:03.532908 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 05:02:03.533344 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":542,"threads_started":1}
I20260809 05:02:03.536623 2434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.243.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.243.252@KRBTEST.COM" created.
Aug 09 05:02:03 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
I20260809 05:02:03.551328 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.537359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46802)
0809 05:02:03.537537 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.537556 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:02:03.537646 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:02:03.538860 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:02:03.538865 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:02:03.539235 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:02:03.539440 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:03.539450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.540875 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.540880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:03.541739 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:03.541751 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.542857 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.542862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:03.542991 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:03.543566 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:02:03.543584 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:02:03.545660 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:02:03.548759 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:03.548763 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:03.548766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:03.549230 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:03.549640 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:03.549644 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:03.549648 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:03.549811 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:03.550401 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:02:03.550405 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:02:03.550553 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 05:02:03.550692 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 05:02:03.550943 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 05:02:03.551380 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.537709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46802 (local address 127.29.243.253:42549)
0809 05:02:03.538405 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:03.538411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:03.538437 (+ 26us) server_negotiation.cc:407] Connection header received
0809 05:02:03.538478 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:03.538483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:03.538670 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:03.538788 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:03.539609 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.540738 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.541887 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.542684 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.542819 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:03.545896 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:03.545908 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:03.545922 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:03.545957 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:03.548625 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:03.549379 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:03.549387 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:03.549391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:03.549530 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:03.549920 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:03.549924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:03.549926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:03.550275 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:03.550452 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:03.550832 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 05:02:03.551037 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":385,"threads_started":1}
I20260809 05:02:03.554046 2434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:02:03.555233 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.554723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48170)
0809 05:02:03.554909 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.555050 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 05:02:03.555577 2359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
I20260809 05:02:03.557121 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.556789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48182)
0809 05:02:03.556925 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.557022 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 05:02:03.557327 2434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:44801: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.243.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab.
Entry for principal kudu/127.29.243.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab.
I20260809 05:02:03.578800 30671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
/tmp/dist-test-taskDjfSyW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.252:44801
--webserver_interface=127.29.243.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab
--principal=kudu/127.29.243.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.243.212:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.243.254:41707,127.29.243.253:42549,127.29.243.252:44801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:02:03.606629 2434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } attempt: 1
I20260809 05:02:03.607905 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.607425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48188)
0809 05:02:03.607634 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.607742 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 05:02:03.608878 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.608528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48190)
0809 05:02:03.608689 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.608770 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 05:02:03.609110 2434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:44801: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
I20260809 05:02:03.664341 2322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } attempt: 5
Aug 09 05:02:03 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251722, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
I20260809 05:02:03.677306 2434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } attempt: 2
I20260809 05:02:03.678380 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.677875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48192)
0809 05:02:03.678090 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.678189 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 05:02:03.678720 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.665225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.29.243.253:42549)
0809 05:02:03.665422 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:03.665428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:03.665455 (+ 27us) server_negotiation.cc:407] Connection header received
0809 05:02:03.665678 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:03.665682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:03.665768 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:03.665855 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:03.666681 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.667423 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.668572 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.669021 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.669154 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:03.672465 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:03.672472 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:03.672477 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:03.672521 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:03.675961 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:03.676623 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:03.676629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:03.676632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:03.676787 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:03.677117 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:03.677122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:03.677125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:03.677513 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:03.677663 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:03.678186 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 05:02:03.678407 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 05:02:03.678758 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.665149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46816)
0809 05:02:03.665373 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.665394 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:02:03.665534 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:02:03.665913 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:02:03.665918 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:02:03.666322 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:02:03.666515 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:03.666525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.667551 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.667556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:03.668408 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:03.668421 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:03.669198 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:03.669205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:03.669420 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:03.670002 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:02:03.670020 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:02:03.672251 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:02:03.676118 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:03.676122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:03.676125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:03.676506 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:03.676887 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:03.676890 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:03.676892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:03.677026 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:03.677646 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:02:03.677650 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:02:03.677825 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 05:02:03.677978 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 05:02:03.678299 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 05:02:03.679600 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.679238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48204)
0809 05:02:03.679391 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.679476 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 05:02:03.679894 2434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:44801: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
I20260809 05:02:03.680071 2322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:02:03.680946 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.680603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48216)
0809 05:02:03.680756 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.680832 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 05:02:03.681795 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.681488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48228)
0809 05:02:03.681634 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.681696 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 05:02:03.681985 2322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:44801: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
I20260809 05:02:03.733238 2322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } attempt: 1
I20260809 05:02:03.734596 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.734026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48244)
0809 05:02:03.734246 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.734380 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 05:02:03.735598 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.735248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48248)
0809 05:02:03.735418 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.735493 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 05:02:03.735806 2322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:44801: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
I20260809 05:02:03.771103 2434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } attempt: 3
I20260809 05:02:03.772264 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.771784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48264)
0809 05:02:03.771981 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.772087 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 05:02:03.773345 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.772937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48280)
0809 05:02:03.773172 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.773237 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260809 05:02:03.773561 2434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:44801: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
I20260809 05:02:03.804033 2322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } attempt: 2
I20260809 05:02:03.805296 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.804791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48286)
0809 05:02:03.805028 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.805121 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 05:02:03.806334 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.806006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48296)
0809 05:02:03.806156 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.806225 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 05:02:03.806550 2322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:44801: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
I20260809 05:02:03.912757 2322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } attempt: 3
I20260809 05:02:03.913954 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.913466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48300)
0809 05:02:03.913674 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.913765 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 05:02:03.915038 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.914653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48312)
0809 05:02:03.914853 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.914927 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260809 05:02:03.915249 2322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:44801: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
I20260809 05:02:03.916725 2434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } attempt: 4
I20260809 05:02:03.917745 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.917254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48314)
0809 05:02:03.917491 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.917587 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 05:02:03.918653 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:03.918331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48328)
0809 05:02:03.918481 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:03.918553 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 05:02:03.918884 2434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.243.252:44801: Network error: Client connection negotiation failed: client connection to 127.29.243.252:44801: connect: Connection refused (error 111)
W20260809 05:02:03.926776 2445 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:02:03.927119 2445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:02:03.927198 2445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:02:03.927312 2445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:02:03.934559 2445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:02:03.934648 2445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:02:03.934715 2445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:02:03.934785 2445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:02:03.934870 2445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:02:03.944744 2445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.243.212:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.243.254:41707,127.29.243.253:42549,127.29.243.252:44801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/krb5kdc/kudu_127.29.243.252.keytab
--principal=kudu/127.29.243.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.243.252:44801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.243.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 05:02:03.946065 2445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:02:03.947497 2445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:02:03.959893 2457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:02:03.960088 2455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:02:03.960392 2463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:02:03.961104 2445 server_base.cc:1034] running on GCE node
I20260809 05:02:03.961862 2445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:02:03.963302 2445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:02:03.964546 2445 hybrid_clock.cc:648] HybridClock initialized: now 1786251723964470 us; error 53 us; skew 500 ppm
I20260809 05:02:03.964879 2445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:02:03 dist-test-slave-21b9 krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:02:03.968056 2445 init.cc:377] Logged in from keytab as kudu/127.29.243.252@KRBTEST.COM (short username kudu)
I20260809 05:02:03.969507 2445 webserver.cc:460] Webserver started at http://127.29.243.252:34209/ using document root <none> and password file <none>
I20260809 05:02:03.970256 2445 fs_manager.cc:359] Metadata directory not provided
I20260809 05:02:03.970366 2445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:02:03.970688 2445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:02:03.973799 2445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/data/instance:
uuid: "dc852b08da124159babf1a2dce6a75eb"
format_stamp: "Formatted at 2026-08-09 05:02:03 on dist-test-slave-21b9"
server_key: "8608c28f3db1b424d4c83cde83349eab"
server_key_iv: "5b76988aa9435adaad33aa6ff458f644"
server_key_version: "encryptionkey@0"
I20260809 05:02:03.974592 2445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/wal/instance:
uuid: "dc852b08da124159babf1a2dce6a75eb"
format_stamp: "Formatted at 2026-08-09 05:02:03 on dist-test-slave-21b9"
server_key: "8608c28f3db1b424d4c83cde83349eab"
server_key_iv: "5b76988aa9435adaad33aa6ff458f644"
server_key_version: "encryptionkey@0"
I20260809 05:02:03.979838 2445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 05:02:03.983345 2476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:02:03.984720 2445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:02:03.984915 2445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/data,/tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/wal
uuid: "dc852b08da124159babf1a2dce6a75eb"
format_stamp: "Formatted at 2026-08-09 05:02:03 on dist-test-slave-21b9"
server_key: "8608c28f3db1b424d4c83cde83349eab"
server_key_iv: "5b76988aa9435adaad33aa6ff458f644"
server_key_version: "encryptionkey@0"
I20260809 05:02:03.985128 2445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:02:04.018858 2445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:02:04.023082 2445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:02:04.023379 2445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:02:04.040998 2445 rpc_server.cc:307] RPC server started. Bound to: 127.29.243.252:44801
I20260809 05:02:04.041020 2551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.243.252:44801 every 8 connection(s)
I20260809 05:02:04.042407 2445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/data/info.pb
I20260809 05:02:04.046227 2553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:02:04.049662 30671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu as pid 2445
I20260809 05:02:04.049865 30671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDjfSyW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786251347713215-30671-0/minicluster-data/master-2/wal/instance
I20260809 05:02:04.050424 30671 external_mini_cluster.cc:1428] Setting key ac22e8a5179b9e0efee216f4a91eb481
I20260809 05:02:04.054141 2553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:02:04 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
Aug 09 05:02:04 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:02:04.072327 2557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.057322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.29.243.254:41707)
0809 05:02:04.057798 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.057804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.058242 (+ 438us) server_negotiation.cc:407] Connection header received
0809 05:02:04.058448 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.058452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.058521 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.058618 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.059789 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.060487 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.061925 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.062576 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.062713 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.065877 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:04.065884 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:04.065890 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:04.065927 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:04.069093 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.069884 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.069889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.069892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.070041 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.070480 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.070484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.070487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.070822 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:04.070972 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.071548 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.071745 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":234,"threads_started":1}
I20260809 05:02:04.073132 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.052345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.29.243.252:44801)
0809 05:02:04.053153 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.053165 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.053210 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:02:04.054824 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.054849 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.055333 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.055718 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.057344 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.059087 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.060375 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.061078 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.061268 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.065551 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:04.065559 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:04.065570 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:04.065600 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:04.068934 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.069794 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.069801 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.069803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.070038 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.070441 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.070446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.070449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.071127 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:04.071419 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.071840 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.072079 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":602,"thread_start_us":363,"threads_started":1}
I20260809 05:02:04.073163 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.057414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.254:41707 (local address 127.0.0.1:39564)
0809 05:02:04.058103 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:04.058145 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 05:02:04.058292 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:02:04.058705 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:02:04.058712 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:02:04.059394 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:02:04.059644 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:04.059653 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.060646 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.060650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:04.061758 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:04.061773 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.062777 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.062783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:04.062902 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.063436 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:02:04.063458 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:02:04.065687 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:02:04.069220 (+ 3533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:04.069232 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:04.069244 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:04.069774 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:04.070159 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:04.070164 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:04.070167 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:04.070375 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:04.070938 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:02:04.070944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:02:04.071134 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 05:02:04.071449 (+ 315us) client_negotiation.cc:240] Negotiation successful
0809 05:02:04.071921 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":24,"thread_start_us":342,"threads_started":1}
I20260809 05:02:04.074827 2553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:02:04 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
Aug 09 05:02:04 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.254@KRBTEST.COM
I20260809 05:02:04.089181 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.075768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46820 (local address 127.29.243.253:42549)
0809 05:02:04.075925 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.075929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.075944 (+ 15us) server_negotiation.cc:407] Connection header received
0809 05:02:04.075994 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.075997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.076056 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.076142 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.076932 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.078024 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.079593 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.080223 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.080352 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.083547 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:04.083552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:04.083555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:04.083589 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:04.086348 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.087122 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.087128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.087131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.087332 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.087796 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.087800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.087802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.088083 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:04.088203 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.088668 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.088904 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 05:02:04.089222 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.075541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.253:42549 (local address 127.0.0.1:46820)
0809 05:02:04.075688 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:04.075704 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:02:04.075803 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:02:04.076198 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:02:04.076203 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:02:04.076604 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:02:04.076772 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:04.076784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.078162 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.078168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:04.079460 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:04.079472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.080344 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.080347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:04.080494 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.081003 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:02:04.081020 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:02:04.083341 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:02:04.086484 (+ 3143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:04.086489 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:04.086494 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:04.086987 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:04.087469 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:04.087474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:04.087478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:04.087676 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:04.088211 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:02:04.088216 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:02:04.088386 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 05:02:04.088561 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 05:02:04.088878 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 05:02:04.090416 2553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:02:04.091318 2557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.076311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39572 (local address 127.29.243.254:41707)
0809 05:02:04.076495 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.076501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.076517 (+ 16us) server_negotiation.cc:407] Connection header received
0809 05:02:04.076581 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.076585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.076662 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.076761 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.078162 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.079177 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.080790 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.081278 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.081368 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.085663 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:04.085668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:04.085671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:04.085701 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:04.088707 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.089455 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.089461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.089465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.089595 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.089934 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.089940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.089944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.090228 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:04.090470 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.090782 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.090991 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:02:04.092453 2322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } attempt: 4
Aug 09 05:02:04 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.252@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
Aug 09 05:02:04 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251722, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.254@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
Aug 09 05:02:04 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.243.253@KRBTEST.COM
I20260809 05:02:04.105595 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.091056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48342)
0809 05:02:04.091203 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:04.091221 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:02:04.091364 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:02:04.091995 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:02:04.091999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:02:04.092400 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:02:04.092573 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:04.092585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.093817 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.093822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:04.095184 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:04.095198 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.095972 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.095977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:04.096162 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.096697 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:02:04.096720 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:02:04.099328 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:02:04.102938 (+ 3610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:04.102943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:04.102947 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:04.103489 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:04.103891 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:04.103896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:04.103900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:04.104058 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:04.104574 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:02:04.104579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:02:04.104717 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 05:02:04.104875 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 05:02:04.105165 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 05:02:04.105635 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.091452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48342 (local address 127.29.243.252:44801)
0809 05:02:04.091615 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.091620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.091634 (+ 14us) server_negotiation.cc:407] Connection header received
0809 05:02:04.091816 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.091819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.091874 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.091942 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.092717 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.093665 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.095344 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.095833 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.095948 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.099538 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:04.099544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:04.099549 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:04.099584 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:04.102807 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.103621 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.103626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.103630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.103788 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.104162 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.104167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.104171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.104440 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:04.104611 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.105000 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.105317 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":41}
I20260809 05:02:04.107940 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.092945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48356)
0809 05:02:04.093112 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:04.093128 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:02:04.093223 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:02:04.094416 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:02:04.094421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:02:04.094798 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:02:04.094944 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:04.094952 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.096574 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.096580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:04.097479 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:04.097490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.098199 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.098205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:04.098379 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.098851 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:02:04.098874 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:02:04.101213 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:02:04.105041 (+ 3828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:04.105046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:04.105050 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:04.105543 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:04.106206 (+ 663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:04.106211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:04.106214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:04.106378 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:04.107028 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:02:04.107034 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:02:04.107222 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 05:02:04.107375 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 05:02:04.107631 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 05:02:04.108978 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.093378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48356 (local address 127.29.243.252:44801)
0809 05:02:04.094170 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.094174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.094190 (+ 16us) server_negotiation.cc:407] Connection header received
0809 05:02:04.094223 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.094226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.094275 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.094357 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.095101 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.096457 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.097611 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.098064 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.098167 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.101442 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:04.101449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:04.101453 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:04.101489 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:04.104867 (+ 3378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.105864 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.105869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.105872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.106048 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.106508 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.106513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.106516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.106869 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:04.108326 (+ 1457us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.108449 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.108639 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":698,"thread_start_us":523,"threads_started":1}
I20260809 05:02:04.110268 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.092953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46834 (local address 127.29.243.253:42549)
0809 05:02:04.093417 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.093421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.093438 (+ 17us) server_negotiation.cc:407] Connection header received
0809 05:02:04.093485 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.093489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.093568 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.093667 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.095173 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.096262 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.098212 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.098896 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.098991 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.103765 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:04.103772 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:04.103776 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:04.103815 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:04.107008 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.107848 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.107854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.107857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.108056 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.108536 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.108541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.108545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.108866 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:04.109637 (+ 771us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.109758 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.109972 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352}
I20260809 05:02:04.111826 2553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb: Bootstrap starting.
I20260809 05:02:04.113646 2322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e: Bootstrap starting.
I20260809 05:02:04.115476 30671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:02:04.115684 2553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb: Neither blocks nor log segments found. Creating new log.
I20260809 05:02:04.116789 2553 log.cc:824] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb: Log is configured to *not* fsync() on all Append() calls
I20260809 05:02:04.117215 2322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e: Neither blocks nor log segments found. Creating new log.
I20260809 05:02:04.118505 2322 log.cc:824] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e: Log is configured to *not* fsync() on all Append() calls
I20260809 05:02:04.120764 2553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb: No bootstrap required, opened a new log
I20260809 05:02:04.121469 2322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e: No bootstrap required, opened a new log
I20260809 05:02:04.118475 2278 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "563e8f1dbe46485b8eb7158102ccdd8e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39572
I20260809 05:02:04.126485 2322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } }
I20260809 05:02:04.126803 2322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 563e8f1dbe46485b8eb7158102ccdd8e, State: Initialized, Role: FOLLOWER
I20260809 05:02:04.127256 2322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } }
I20260809 05:02:04.128227 2565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } } }
I20260809 05:02:04.128626 2565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:02:04.129410 2322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:02:04.129518 2278 raft_consensus.cc:482] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 05:02:04.129726 2278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:02:04.131403 2553 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } }
I20260809 05:02:04.131817 2553 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc852b08da124159babf1a2dce6a75eb, State: Initialized, Role: FOLLOWER
I20260809 05:02:04.132298 2553 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } }
I20260809 05:02:04.133118 2278 raft_consensus.cc:504] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } }
I20260809 05:02:04.133267 2573 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } } }
I20260809 05:02:04.133893 2573 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:02:04.134493 2553 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:02:04.135850 2515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dc852b08da124159babf1a2dce6a75eb"
I20260809 05:02:04.136344 2515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:02:04.139932 2515 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 563e8f1dbe46485b8eb7158102ccdd8e in term 1.
I20260809 05:02:04.140354 2278 leader_election.cc:288] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [CANDIDATE]: Term 1 election: Requested vote from peers d2c34298aed949d798eabf0c66bce14a (127.29.243.253:42549), dc852b08da124159babf1a2dce6a75eb (127.29.243.252:44801)
I20260809 05:02:04.140837 2235 leader_election.cc:302] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 563e8f1dbe46485b8eb7158102ccdd8e, dc852b08da124159babf1a2dce6a75eb; no voters:
I20260809 05:02:04.141386 2565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:02:04.154417 2565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [term 1 LEADER]: Becoming Leader. State: Replica: 563e8f1dbe46485b8eb7158102ccdd8e, State: Running, Role: LEADER
I20260809 05:02:04.155722 2565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } }
I20260809 05:02:04.172078 2579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 563e8f1dbe46485b8eb7158102ccdd8e. Latest consensus state: current_term: 1 leader_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } } }
I20260809 05:02:04.172541 2579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [sys.catalog]: This master's current role is: LEADER
I20260809 05:02:04.186295 2602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260809 05:02:04.190981 2242 tablet.cc:2369] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:02:04.196419 2602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:02:04.206427 2557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.166000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.29.243.254:41707)
0809 05:02:04.166174 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.166181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.166594 (+ 413us) server_negotiation.cc:407] Connection header received
0809 05:02:04.167169 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.167176 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.167291 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.167373 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.171863 (+ 4490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.173030 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.176309 (+ 3279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.177007 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.177108 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.182849 (+ 5741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:04.182858 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:04.182863 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:04.182917 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:04.187032 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.188456 (+ 1424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.188463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.188467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.190614 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.191367 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.191374 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.191377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.191775 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:04.192147 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.205688 (+ 13541us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.205998 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 05:02:04.218072 2434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } attempt: 5
I20260809 05:02:04.196792 2211 master_runner.cc:186] Error getting master registration for 127.29.243.253:42549: OK, instance_id { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" instance_seqno: 1786251723492857 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 05:02:04.220734 2211 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 05:02:04.208199 2445 master_runner.cc:186] Error getting master registration for 127.29.243.253:42549: OK, instance_id { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" instance_seqno: 1786251723492857 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 05:02:04.226859 2445 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 05:02:04.228144 2621 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:02:04.228271 2621 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 05:02:04 dist-test-slave-21b9 krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786251723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.243.253@KRBTEST.COM for kudu/127.29.243.252@KRBTEST.COM
I20260809 05:02:04.238611 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.167088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46850 (local address 127.29.243.253:42549)
0809 05:02:04.167304 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.167311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.184658 (+ 17347us) server_negotiation.cc:407] Connection header received
0809 05:02:04.188306 (+ 3648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.188313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.188417 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.188495 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.218942 (+ 30447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.220075 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.222648 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.223331 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.223453 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.231567 (+ 8114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:04.231573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:04.231577 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:04.231611 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:04.235107 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.235969 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.235974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.235978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.236160 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.236615 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.236620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.236624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.236990 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:04.237236 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.237892 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.238190 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 05:02:04.242116 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.170977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.29.243.252:44801)
0809 05:02:04.171217 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.171224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.185564 (+ 14340us) server_negotiation.cc:407] Connection header received
0809 05:02:04.189516 (+ 3952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.189523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.189621 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.189719 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.218778 (+ 29059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.224691 (+ 5913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.226552 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.227248 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.227375 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.232666 (+ 5291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:04.232673 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:04.232677 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:04.232717 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:04.237787 (+ 5070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.239733 (+ 1946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.239739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.239742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.239920 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.240375 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.240380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.240384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.240756 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:04.240978 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.241542 (+ 564us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.241773 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 05:02:04.243224 2515 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [term 1 FOLLOWER]: Refusing update from remote peer 563e8f1dbe46485b8eb7158102ccdd8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:02:04.244418 2565 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:02:04.247222 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.218709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.243.252:44801 (local address 127.0.0.1:48368)
0809 05:02:04.218919 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 05:02:04.218939 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:02:04.219069 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:02:04.219763 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:02:04.219768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:02:04.220288 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:02:04.220478 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:04.220489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.224600 (+ 4111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.224605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:04.226020 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:02:04.226035 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.227302 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.227308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:02:04.227556 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.228500 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:02:04.228522 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:02:04.231318 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:02:04.240830 (+ 9512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:04.240835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:04.240858 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:04.241585 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:04.242975 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:02:04.243001 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:02:04.243005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:02:04.243325 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:02:04.245155 (+ 1830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:02:04.245160 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:02:04.245342 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 05:02:04.245475 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 05:02:04.246835 (+ 1360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 05:02:04.248569 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.218735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.29.243.252:44801)
0809 05:02:04.219326 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.219332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.219349 (+ 17us) server_negotiation.cc:407] Connection header received
0809 05:02:04.219398 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.219403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.219509 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.219599 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.220630 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.224464 (+ 3834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.226187 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.226895 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.227038 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.231531 (+ 4493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:02:04.231538 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:02:04.231543 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:02:04.231581 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:02:04.240603 (+ 9022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.242538 (+ 1935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.242550 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.242553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.242770 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:02:04.243644 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:02:04.243649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:02:04.243652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:02:04.244954 (+ 1302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:02:04.247830 (+ 2876us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.248010 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.248226 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":441}
I20260809 05:02:04.255801 2434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a: Bootstrap starting.
I20260809 05:02:04.260200 2434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a: Neither blocks nor log segments found. Creating new log.
I20260809 05:02:04.261524 2434 log.cc:824] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a: Log is configured to *not* fsync() on all Append() calls
I20260809 05:02:04.269801 2573 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 563e8f1dbe46485b8eb7158102ccdd8e. Latest consensus state: current_term: 1 leader_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } } }
I20260809 05:02:04.270118 2573 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:02:04.270992 2434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a: No bootstrap required, opened a new log
I20260809 05:02:04.276371 2573 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } } }
I20260809 05:02:04.276613 2573 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:02:04.277066 2624 mvcc.cc:205] Tried to move back new op lower bound from 7316487062416211968 to 7316487062153928704. Current Snapshot: MvccSnapshot[applied={T|T < 7316487062416211968}]
I20260809 05:02:04.278210 2565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } } }
I20260809 05:02:04.278503 2565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [sys.catalog]: This master's current role is: LEADER
I20260809 05:02:04.283684 2602 catalog_manager.cc:1347] Generated new cluster ID: a1b8576d915a4f2aadbbb5e7d7c83686
I20260809 05:02:04.283790 2602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:02:04.291208 2434 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } }
I20260809 05:02:04.291802 2434 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c34298aed949d798eabf0c66bce14a, State: Initialized, Role: FOLLOWER
I20260809 05:02:04.292415 2434 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } }
I20260809 05:02:04.307848 2633 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } } }
I20260809 05:02:04.308344 2633 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:02:04.309535 2434 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:02:04.309595 2396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d2c34298aed949d798eabf0c66bce14a"
I20260809 05:02:04.310105 2396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:02:04.313675 2396 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 563e8f1dbe46485b8eb7158102ccdd8e in term 1.
I20260809 05:02:04.314724 2395 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [term 1 FOLLOWER]: Refusing update from remote peer 563e8f1dbe46485b8eb7158102ccdd8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:02:04.316298 2579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:02:04.331666 2602 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:02:04.333458 2602 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:02:04.348351 2565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } } }
I20260809 05:02:04.348662 2565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e [sys.catalog]: This master's current role is: LEADER
I20260809 05:02:04.348495 2633 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 563e8f1dbe46485b8eb7158102ccdd8e. Latest consensus state: current_term: 1 leader_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } } }
I20260809 05:02:04.348843 2633 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:02:04.350914 2640 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563e8f1dbe46485b8eb7158102ccdd8e" member_type: VOTER last_known_addr { host: "127.29.243.254" port: 41707 } } peers { permanent_uuid: "d2c34298aed949d798eabf0c66bce14a" member_type: VOTER last_known_addr { host: "127.29.243.253" port: 42549 } } peers { permanent_uuid: "dc852b08da124159babf1a2dce6a75eb" member_type: VOTER last_known_addr { host: "127.29.243.252" port: 44801 } } }
I20260809 05:02:04.351444 2640 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:02:04.370787 2602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 563e8f1dbe46485b8eb7158102ccdd8e: Generated new TSK 0
I20260809 05:02:04.371977 2602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:02:04.392457 2653 catalog_manager.cc:1259] Loaded cluster ID: a1b8576d915a4f2aadbbb5e7d7c83686
I20260809 05:02:04.392633 2653 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a: loading cluster ID for follower catalog manager: success
I20260809 05:02:04.410279 2653 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a: acquiring CA information for follower catalog manager: success
I20260809 05:02:04.413203 2653 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d2c34298aed949d798eabf0c66bce14a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 05:02:04.528541 2557 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:04.505320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39578 (local address 127.29.243.254:41707)
0809 05:02:04.505552 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.505558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.525764 (+ 20206us) server_negotiation.cc:407] Connection header received
0809 05:02:04.526335 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.526343 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.526437 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.526532 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.528265 (+ 1733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39578: BlockingRecv error: recv got EOF from 127.0.0.1:39578 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 05:02:04.531502 2556 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:04.512166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.29.243.252:44801)
0809 05:02:04.512394 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.512402 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.529493 (+ 17091us) server_negotiation.cc:407] Connection header received
0809 05:02:04.529953 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.529959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.530063 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.530135 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.531210 (+ 1075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48378: BlockingRecv error: recv got EOF from 127.0.0.1:48378 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260809 05:02:04.534787 2442 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:04.532688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.29.243.253:42549)
0809 05:02:04.532901 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.532907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.533535 (+ 628us) server_negotiation.cc:407] Connection header received
0809 05:02:04.533680 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.533685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.533769 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.533877 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.534589 (+ 712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46852: BlockingRecv error: recv got EOF from 127.0.0.1:46852 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 05:02:04.584035 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.565368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46868 (local address 127.29.243.253:42549)
0809 05:02:04.565563 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.565570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.576159 (+ 10589us) server_negotiation.cc:407] Connection header received
0809 05:02:04.576733 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.576741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.576972 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.577121 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:04.577898 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.579025 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.580998 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.581666 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.581754 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.582502 (+ 748us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:04.583190 (+ 688us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:04.583323 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.583576 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.583675 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 05:02:04.642521 2556 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:04.629388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48392 (local address 127.29.243.252:44801)
0809 05:02:04.629595 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.629602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.640205 (+ 10603us) server_negotiation.cc:407] Connection header received
0809 05:02:04.640779 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.640786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.640946 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.641036 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.642226 (+ 1190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48392: BlockingRecv error: recv got EOF from 127.0.0.1:48392 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 05:02:04.697824 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.678981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46870 (local address 127.29.243.253:42549)
0809 05:02:04.679227 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.679234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.690351 (+ 11117us) server_negotiation.cc:407] Connection header received
0809 05:02:04.690764 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.690771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.690915 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.690999 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:04.691766 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.692947 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.695366 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.696100 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.696206 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.696871 (+ 665us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:04.697068 (+ 197us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:04.697152 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.697390 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.697487 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260809 05:02:04.753748 2556 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:04.737108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.29.243.252:44801)
0809 05:02:04.737335 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.737343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.747998 (+ 10655us) server_negotiation.cc:407] Connection header received
0809 05:02:04.752151 (+ 4153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.752158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.752275 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.752352 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.753485 (+ 1133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48396: BlockingRecv error: recv got EOF from 127.0.0.1:48396 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:02:04.796481 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.789454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46884 (local address 127.29.243.253:42549)
0809 05:02:04.789676 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.789683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.790264 (+ 581us) server_negotiation.cc:407] Connection header received
0809 05:02:04.790443 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.790449 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.790547 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.790657 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:04.791144 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.792361 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.794259 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.794935 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.795074 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.795587 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:04.795754 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:04.795851 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.796005 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.796093 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
W20260809 05:02:04.811534 2556 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:04.808995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48406 (local address 127.29.243.252:44801)
0809 05:02:04.809229 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.809236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.809798 (+ 562us) server_negotiation.cc:407] Connection header received
0809 05:02:04.810026 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.810033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.810175 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.810299 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.811228 (+ 929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48406: BlockingRecv error: recv got EOF from 127.0.0.1:48406 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 05:02:04.835490 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.828996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46886 (local address 127.29.243.253:42549)
0809 05:02:04.829256 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.829263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.829763 (+ 500us) server_negotiation.cc:407] Connection header received
0809 05:02:04.830097 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.830104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.830235 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.830342 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:04.830827 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.832058 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.833522 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.834217 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.834349 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.834705 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:04.834844 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:04.834950 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.835071 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.835161 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
W20260809 05:02:04.843722 2556 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:04.841381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.29.243.252:44801)
0809 05:02:04.841672 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.841678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.842146 (+ 468us) server_negotiation.cc:407] Connection header received
0809 05:02:04.842324 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.842346 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.842419 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.842499 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.843473 (+ 974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48412: BlockingRecv error: recv got EOF from 127.0.0.1:48412 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
I20260809 05:02:04.861554 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.855586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46896 (local address 127.29.243.253:42549)
0809 05:02:04.855803 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.855810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.856204 (+ 394us) server_negotiation.cc:407] Connection header received
0809 05:02:04.856377 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.856383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.856480 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.856602 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:04.857042 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.858177 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.859574 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.860239 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.860370 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.860735 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:04.860913 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:04.860999 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.861099 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.861181 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 05:02:04.869490 2556 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:04.867564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48418 (local address 127.29.243.252:44801)
0809 05:02:04.867756 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.867762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.868132 (+ 370us) server_negotiation.cc:407] Connection header received
0809 05:02:04.868294 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.868300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.868387 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.868484 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.869265 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48418: BlockingRecv error: recv got EOF from 127.0.0.1:48418 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 05:02:04.894750 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.888967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46904 (local address 127.29.243.253:42549)
0809 05:02:04.889176 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.889182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.889501 (+ 319us) server_negotiation.cc:407] Connection header received
0809 05:02:04.889673 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.889678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.889792 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.889911 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:04.890325 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.891540 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.892920 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.893541 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.893654 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.894028 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:04.894155 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:04.894241 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.894351 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.894446 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 05:02:04.902248 2556 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:04.900450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.29.243.252:44801)
0809 05:02:04.900651 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.900657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.901016 (+ 359us) server_negotiation.cc:407] Connection header received
0809 05:02:04.901172 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.901177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.901266 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.901358 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.902047 (+ 689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48422: BlockingRecv error: recv got EOF from 127.0.0.1:48422 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 05:02:04.942556 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:04.937581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46906 (local address 127.29.243.253:42549)
0809 05:02:04.937750 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.937755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.938155 (+ 400us) server_negotiation.cc:407] Connection header received
0809 05:02:04.938349 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.938354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.938462 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.938563 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:04.938916 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.939746 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.941041 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:04.941480 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:04.941575 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:04.941950 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:04.942082 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:04.942161 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:04.942248 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 05:02:04.942312 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 05:02:04.950052 2556 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:04.948283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.29.243.252:44801)
0809 05:02:04.948512 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:04.948518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:04.948735 (+ 217us) server_negotiation.cc:407] Connection header received
0809 05:02:04.948876 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:04.948879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:04.948965 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:04.949058 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:04.949785 (+ 727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48438: BlockingRecv error: recv got EOF from 127.0.0.1:48438 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 05:02:05.022643 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:05.017706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46920 (local address 127.29.243.253:42549)
0809 05:02:05.017947 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:05.017954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:05.018212 (+ 258us) server_negotiation.cc:407] Connection header received
0809 05:02:05.018384 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:05.018387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:05.018475 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:05.018581 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:05.018941 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.019751 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.021071 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.021463 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.021564 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:05.021923 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:05.022037 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:05.022104 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:05.022277 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 05:02:05.022339 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 05:02:05.030165 2556 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:05.028171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48454 (local address 127.29.243.252:44801)
0809 05:02:05.028345 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:05.028351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:05.028717 (+ 366us) server_negotiation.cc:407] Connection header received
0809 05:02:05.028945 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:05.028950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:05.029028 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:05.029130 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:05.029930 (+ 800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48454: BlockingRecv error: recv got EOF from 127.0.0.1:48454 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 05:02:05.166520 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:05.161851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46922 (local address 127.29.243.253:42549)
0809 05:02:05.162092 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:05.162099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:05.162407 (+ 308us) server_negotiation.cc:407] Connection header received
0809 05:02:05.162593 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:05.162597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:05.162673 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:05.162755 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:05.163105 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.163831 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.165067 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.165473 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.165572 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:05.165897 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:05.166005 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:05.166064 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:05.166159 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 05:02:05.166218 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 05:02:05.173476 2556 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:02:05.171877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48456 (local address 127.29.243.252:44801)
0809 05:02:05.172050 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:05.172054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:05.172395 (+ 341us) server_negotiation.cc:407] Connection header received
0809 05:02:05.172550 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:05.172554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:05.172624 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:05.172704 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:02:05.173281 (+ 577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48456: BlockingRecv error: recv got EOF from 127.0.0.1:48456 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 05:02:05.230424 2621 catalog_manager.cc:1259] Loaded cluster ID: a1b8576d915a4f2aadbbb5e7d7c83686
I20260809 05:02:05.230540 2621 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb: loading cluster ID for follower catalog manager: success
I20260809 05:02:05.233192 2621 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb: acquiring CA information for follower catalog manager: success
I20260809 05:02:05.235142 2621 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dc852b08da124159babf1a2dce6a75eb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 05:02:05.441916 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:05.436549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46930 (local address 127.29.243.253:42549)
0809 05:02:05.436779 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:05.436786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:05.436910 (+ 124us) server_negotiation.cc:407] Connection header received
0809 05:02:05.437134 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:05.437138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:05.437211 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:05.437295 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:05.437779 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.438703 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.440264 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.440680 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.440794 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:05.441200 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:05.441336 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:05.441413 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:05.441564 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 05:02:05.441649 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:02:05.453518 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:05.447751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.29.243.252:44801)
0809 05:02:05.448011 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:05.448018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:05.448277 (+ 259us) server_negotiation.cc:407] Connection header received
0809 05:02:05.448478 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:05.448484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:05.448657 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:05.448766 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:05.449226 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.449942 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.451286 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.451706 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.451840 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:05.452355 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:05.452897 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:05.453064 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:05.453161 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 05:02:05.453252 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 05:02:05.467566 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:05.460324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.29.243.252:44801)
0809 05:02:05.460518 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:05.460524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:05.461455 (+ 931us) server_negotiation.cc:407] Connection header received
0809 05:02:05.461706 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:05.461712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:05.461808 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:05.461873 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:05.462506 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.463525 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.465435 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.466058 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.466197 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:05.466732 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:05.466862 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:05.466966 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:05.467125 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 05:02:05.467208 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 05:02:05.469041 2854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:05.460267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39586 (local address 127.29.243.254:41707)
0809 05:02:05.460888 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:05.460894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:05.461341 (+ 447us) server_negotiation.cc:407] Connection header received
0809 05:02:05.461727 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:05.461735 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:05.461994 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:05.462178 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:05.463151 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.464233 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.466378 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.466864 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.466983 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:05.467969 (+ 986us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:05.468326 (+ 357us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:05.468420 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:05.468676 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 05:02:05.468765 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":356,"threads_started":1}
I20260809 05:02:05.469290 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:02:05.461728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46942 (local address 127.29.243.253:42549)
0809 05:02:05.461928 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 05:02:05.461933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:02:05.463146 (+ 1213us) server_negotiation.cc:407] Connection header received
0809 05:02:05.463421 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:02:05.463428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:02:05.463529 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:02:05.463635 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:02:05.464525 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.465264 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.467105 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:02:05.467753 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:02:05.467871 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:02:05.468439 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:02:05.468582 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:02:05.468673 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 05:02:05.468887 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 05:02:05.468982 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 05:02:05.473596 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 2211
I20260809 05:02:05.475093 2314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:02:05.591110 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 2211
I20260809 05:02:05.625265 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 2331
I20260809 05:02:05.626448 2424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:02:05.733204 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 2331
I20260809 05:02:05.761780 30671 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu pid 2445
I20260809 05:02:05.762971 2544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:02:05.869371 30671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDjfSyW/build/asan/bin/kudu with pid 2445
2026-08-09T05:02:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4934 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10093 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (377986 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST